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1.  The  objective  of  this  study  was  to.  identify  and  analyze  curriculum 
development  policies  and  procedures  used  by  the  Army,  Navy,  and  Air  Force 

to  determine  the  subject  matter  for  technical  training  courses.  It  was  con¬ 
ducted  in  1966  as  part  of  the  Consolidated  Training  and  Education  Program, 
a  special  study  directed  by  the  Assistant  Secretary  of  Defense  (Manpower 
and  Reserve  Affairs).  The  Office  of  the  Chief  of  Research  and  Development, 
Department  of  the  Army,  monitored  the  research  in  this  report. 

2.  A  model  for  the  curriculum  development  sequence  was  defined  from 
training  research  findings  and  practices.  This  model  consists  of  the  steps: 
(a)  analyze  the  system,  (b)  develop  task  inventories,  (c)  develop  a  Job  model, 
(d)  analyze  the  tasks,  (e)  derive  training  objectives,  (f)  develop  the 
training  program,  and  (g)  monitor  the  trained  product  and  modify  the  curri¬ 
culum.  Key  training  headquarters  and  installations  were  visited  and  the 
procedures  being  used  in  the  three  services  at  the  time  of  the  study  were 
compared  with  the  model  to  identify  ways  of  improving  existing  practices. 
Although  policies  and  procedures  used  by  the  services  have  changed  since 

the  time  of  the  study,  this  report  is  being  issued  to  provide  a  record  of 
the  methodology  used  in  the  study  and  of  the  procedures  then  being  used  by 
the  services  for  curriculum  development.  The  record  will  facilitate  future 
studies  of  curriculum  development  procedures  of  the  services  and  provide  in¬ 
formation  for  comparison. 

3.  The  seven  research  findings  and  practices  mentioned  in  paragraph  two 
were  used  as  supportive  research  data  for  a  committee  that  produced  US 
Continental  Army  Command  Regulation  350-100-1,  Systems  Engineering  of 
Training.  The  foundation  for  the  regulation  is  task  and  skill  analysis. 

After  these  factors  are  determined,  training  objectives  can  be  identified 
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and  nev  course  revision  can  be  instituted  that  will  be  closely  Integrated 
with  Innovations  in  equiptsent,  tactics  and  organisation. 

4.  This  report  will  be  of  Interest  to  personnel  concerned  with  curriculum 
development  and  research  in  military  training  and  to  personnel  Interested 
in  vocational  and  technical  education  and  research. 

FOR  THE  CHIEF  OF  RESEARCH  AND  DEVELOPMENT: 


Chief,  Behavioral 
Sciences  Division 
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FOREWORD 


In  December  1965,  a  memorandum  from  the  Office  of  the  Assistant  Secre¬ 
tary  of  Defense  (Manpower)  on  the  Calendar  Year  1966  Consolidated  Training 
and  Education  Training  Program  (CTEP)  was  addressed  to  the  Secretaries  of 
the  Army,  Navy,  and  Air  Force.  It  stated  in  part: 

...  a  number  of  special  studies  will  be  conducted  under  the  overall  supervision  of  the 
Assistant  Secretary  of  Defense  (Manpower).  The  studies  will  review  the  policies,  proce¬ 
dures,  methodologies,  and  facilities  associated  with  the  conduct  of  formalized  individual 
trnining  programs  in  the  Military  Departments. 

The  Curriculum  Content  Study  Area— one  of  11  subject  areas  in  the  program— 
was  assigned  to  the  Human  Resources  Research  Office.  The  HumRRO  effort  was 
initiated  in  December  1965;  a  report,  describing  the  various  procedures  used  to 
develop  certain  types  of  technical  training  content  in  the  several  military  serv¬ 
ices,  was  submitted  to  the  Assistant  Secretary  of  Defense  (Manpower)  in 
August  1966.  This  Technical  Report  is  based  on  the  report  to  the  Secretary. 

The  curriculum  development  procedures  described  have  been  changed  and 
been  improved  since  the  period  of  study.  This  Technical  Report  is,  therefore, 
presented  not  as  a  report  of  current  status  but  as  of  the  situation  existing  in 
1966.  It  is  being  published  at  this  time  to  provide  a  generally  available  record 
of  the  methodology  used  in  that  study,  and  of  the  study  findings,  to  serve  as  a 
point  of  departure  and  "benchmark"  reference  for  future  studies  of  methods  and 
procedures  for  developing  technical  training  content  in  the  military. 

The  study  was  conducted  under  the  supervision  of  Dr.  J.  Daniel  Lyons, 
Director  of  Research,  HumRRO  Division  No.  1  (System  Operations).  The  imme¬ 
diate  staff  consisted  of  Dr.  Harold  G.  Hunter,  Dr.  Eugene  F.  MacCaslin,  and 
Dr.  Harold  Wagner— all  staff  members  of  Division  1— and  Dr.  Robert  G.  Smith, 
Jr.,  Representative  of  the  Director,  HumRRO,  at  USCONARC,  a  staff  member  of 
the  Director's  office. 

HumRRO  research  for  the  Department  of  the  Army  is  conducted  under 
Contract  DAHC  19-69-C-0018;  Training,  Motivation,  Leadership  Research  is 
conducted  under  Army  Project  2 J062107 A712. 


Meredith  P.  Crawford 
Director 

Human  Resources  Research  Office 
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8UMMARY  AND 


ProbUm 

Tc  succeed  in  training  men  to  meet  the  know’rdge  and  skill  requirements  for  a  particular 
military  job  military  traininq  courses  must  have  content  appropriate  to  the  job.  This  depends 
upon  the  existence  of  effective  procedures  for  developing  and  maintaining  the  appropriate 
instruction  >1  content 

Obj«ctiv«*  of  th«  Study 

A  study  to  identify  and  evaluate  current  procedures  tot  determining  the  content  of  technical 
traininq  courses  was  undertaken  by  'he  Human  Resources  Research  Office  in  1966  The  study 
was  part  of  the  Consolidated  Training  and  Education  Proqram  (CTEP)  supervised  by  the  Assistant 
Secretary  of  Defense  (Manpower)  and  dealt  with  the  Curriculum  Content  Study  Area  of  the  explor¬ 
atory  project 

The  military  services  have  undoubtedly  made  many  changes  and  improvements  in  their 
curriculum  content  procedures  since  that  study  was  made.  This  report  is  presented  not  as  a 
statement  of  current  status  in  content  development  procedures  but  to  make  available  a  record 
of  the  range  and  type  of  procedures  in  use  by  the  various  services  at  a  particular  time.  It  is 
hoped  that  this  information  will  be  useful  tc  training  and  management  personnel  as  a  record  of 
the  study's  methodology  and  as  a  basis  far  assessing  improvements  that  have  been  made  since 
that  time  and  in  identifying  cr-as  where  further  efforts  for  improvement  n  iqht  be  fruitful. 

The  scope  of  the  study  area  was  confined  to  first-terr  enlisted  technical  tra.ning  particu 
iarly  in  electronics  and  other  technical  fields  in  which  training  costs  are  high  Only  those  train 
tng  procedures  used  by  the  U  S  Continental  Army  Command  the  Bureau  of  Naval  Personnel 
the  Naval  At;  Technical  Training  Command,  llie  Air  Training  Command,  and  facilities  under  thei; 
Cutnmond  were  studied  No  ‘acihties  devoted  exclusively  to  Marine  Corps  training  were  included 
Particular  attention  was  pa.d  to  the  procedures  used  by  the  services  to  gather  information  lor 
revising  training  curricula. 


Approach 

After  a  search  of  the  military  training  literature  HumRRO  constructed  a  moo*!  for  cuftic 
ulutn  development  in  military  training  the  major  feature*  of  which  have  emerged  from.  nodsin 
training  research  The  model  consists  of  seven  steps  to  curriculum  development 
•  U  Conduct  system,  analysis 
!’)  Develop  task  inventory 
■;9)  Develop  10b  model 
(4!  Conduct  task  analysis 
i'S)  Derive  'taming  objectives 
(6?  Develop  framing  j.-ogram 

?!  Monitor  trained  product  and  m-addy  training  cufucuium 

Information  on  current  procedure  -  -  obtained  irons  visits  to  kev  training  installation*  of 

the  Army  Navy  and  At  f  farce  jis  -.sdei  was  compared  with  existing  pt  acedufes 

which  the  military  services  use  to  i  :-pruv«  he-.r  ‘-a.r-.  'ifncula 

Finding* 

The  rornparison  sd  "he  mod*!  process  for  d*vw;*.pr sen*  at  "ramm-a  :ur riculj  with  the  proc 
esses  *s*d  by  the  n ;  1 , ‘  jr y  services  a*  'he  t  cf  'his  survey  indicated  'hat 


(1)  Few  procedures  bearing  an  the  first  lour  steps  ol  the  --.odel  were  in  effect  in  the 
services  Specifically: 

(a)  None  of  the  services  required  system  analysis  far  traininq  purposes.  The  Air 
Force  required  system  analysis  durinq  system  development. 

(b)  The  Air  Force  and  the  Army  but  not  the  Navy  'equired  task  inventories,  but  not 
that  the  inventories  be  updated. 

(c)  None  of  the  services  required  development  of  a  job  model  that  is  a  composite  of 
!>■(.  requirements  of  the  joe 

(d)  None  of  the  services  required  task  analysis  for  curriculum  development 

12)  Ail  the  services  recoqnized  that  traininq  objectives  should  be  relevant  to  the  job. 
ana  all  provided  guidance  on  warding  and  format.  However,  there  were  no  directives  for  collectinq 
and  analyzinq  job  information  to  make  objectives  as  specific  as  possible 

(3)  Procedures  for  develop. nq  traininq  proqrams  were  not  fully  effective  because  ccurse 
objectives  had  not  been  fully  specified.  Information  on  the  capability  demanded  of  the  qraduate 
was  also  needed  for  more  effective  development  of  traininq  proqrams. 

(4)  Inqeneral.  evaluation  practices  of  the  services  did  not  assess  traininq  effectiveness 
The  Air  Force  had  the  only  standard  of  qraduate  capability,  and  also  was  the  only  service  that 
conducted  field  visits  The  other  services  obtained  feedback  on  traininq  effectiveness  mainly 
from  mailed  (questionnaires 

(5)  The  importance  of  traininq  as  a  military  activity  is  indicated  by  the  fact  that  training 
costs  amount  to  6*  a  of  the  Defense  Budqel  The  Air  Fores  was  the  only  setviev  however  that 
offered  a  training  career  field  and  it  accounted  tor  less  than  1*.  of  Ait  Force  strength. 

Conclusions 

(1)  The  results  ol  HumRRO's  research  tn  the  1966  curriculum  content  study  area  indicated 
that  improved  procedures  to  determine  the  adequacy  of  training  content  and  the  means  for  improve¬ 
ment  were  needed  by  the  services.  The  curriculum  model  that  has  emerged  from  training  resemch 
appears  to  offer  a  useful  pattern  of  improvement  far  all  the  services 

(2)  Directives  and  detailed  procedural  guidance  were  needed  for  conducting  system  and  10b 
analysis  and  for  developing  task  inventories  and  job  models.  Curriculum  development  would 
profit  (torn  a  statement  of  criteria  far  allocating  analytic  content  ’o formal  instruction  or  on  the  job 
training  as  well  as  precise  specifications  tar  graduates  of  traininq  courses  Curriculum  develop 
ment  would  also  profit  from  feedback  to  determine  whether  training  programs  end  objectives  satis 
factorily  meet  job  specifications  Field  visits  would  appear  to  be  a  more  effective  means  of 
obtaining  feedback  than  mailed  questionnaires 

(3)  To  improve  the  procedures  the  services  use  m  curriculum  development  they  needed  to 
provide  more  oppesr* unities  fat  career  fields  m  training  opportunities  .n  proportion  to  the  impcr 
'since  of  training  as  a  military  activity 
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The  Process  of  Developing  and 
Improving  Course  Content  for 
Military  Technical  Training 


Chapter  I 

INTRODUCTION 


Problem  and  Objectives 

Military  training  is  oriented  to  produce  the  knowledge  and  skill  requirements 
for  performing  a  particular  job  to  which  an  individual  is  to  be  assigned.  Meeting 
this  objective  depends  upon  the  existence  of  training  course  content  appropriate 
to  the  job,  and  hence  upon  the  existence  of  effective  procedures  for  developing 
and  maintaining  the  appropriate  instructional  content. 

This  report  is  a  description  of  various  procedures  used  to  develop  certain 
types  of  technical  training  content  in  the  various  military  services.  The  material 
was  collected  and  analyzed  in  a  study  undertaken  by  the  Human  Resources 
Research  Office  as  part  of  the  Consolidated  Training  and  Education  Program 
(CTEP)  under  the  supervision  of  the  Assistant  Secretary  of  Defense  (Manpower). 

The  survey  of  procedures  was  made  in  1966  and,  without  question,  the 
several  services  have  made  many  changes  and  improvements  in  their  curricu¬ 
lum  content  procedures  in  the  intervening  period  ’  This  report  is  presented  not 
as  a  statement  of  current  status,  but  to  make  available  to  training  and  manage¬ 
ment  personnel  a  record  of  the  range  and  type  of  procedures  being  used  by  the 
several  services  as  of  a  given  point  in  time.  The  information  may  serve  as  a 
record  of  the  methodology  used  in  the  study  and  as  a  benchmark  against  which 
subsequent  improvements  may  be  viewed,  and  perhaps  as  a  means  of  identifying 
or  suggesting  other  procedural  areas  in  which  further  efforts  toward  improve¬ 
ment  might  be  undertaken. 

Background 

In  December  1965,  a  memorandum  from  the  Office  of  the  Assistant  Secre¬ 
tary  of  Defense  (Manpower)  on  the  Calendar  Year  1966  Consolidated  Training 
and  Education  Program  was  addressed  to  the  Secretaries  of  the  Army,  Navy, 
and  Air  Force.  It  stated  in  part: 

...  a  number  of  special  studies  will  be  conducted  under  the  overall  supervision  of 
the  Assistant  Secretary  of  Defense  (Manpower).  The  studies  will  review  the  policies, 
procedures,  methodologies,  and  facilities  associated  with  the  conduct  of  formalized  indi¬ 
vidual  training  progrums  in  the  Military  Departments. 

The  Curriculum  Content  Study  Area— one  of  11  subject  areas  in  the  program2— 
was  assigned  to  HumRRO.  The  administrative  instructions  defined  the  scope  and 
major  objectives  of  the  study  area: 

This  study  area  will  be  confined  primarily  to  first-term  enlisted  technical  training, 
particularly  in  electronics  and  other  technical  fields  in  which  the  length  of  courses,  cost 
of  schooling,  complexity  of  subject  matter,  and  or  attrition  rate  are  relatively  high.  Less 
intensive  exploration  will  be  given  to  advanced  enlisted  technical  training. 


'An  example  of  action  taken  during  this  period  by  the  li.S.  Continental  Army  Command  is  in  Appendix  A. 
’The  other  subject  areas  were:  requirements  and  programming,  selection  and  classification,  training 
methods,  facilities  utilization,  personnel  utilization,  personnel  research,  professional  education,  service 
academies,  adult  education,  and  pilot  training. 
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The  major  objective  will  be  to  identify  and  evaluate  current  procedures  for  deter¬ 
mining  the  content  of  technical  training  courses  in  terms  of  the  extent  to  which  these  pro¬ 
cedures  produce  courses  in  which  the  content  adequatelv  supports  the  skill  requirements 
of  the  job  to  which  the  individual  is  assigned. 

The  HumRRO  study  of  curriculum  development  and  revision  was  restricted 
to  procedures  employed  by  the  U.S.  Continental  Army  Command,  the  Bureau  of 
Naval  Personnel,  the  Naval  Air  Technical  Training  Command,  the  Air  Training 
Command,  and  facilities  under  their  command.  No  facilities  devoted  exclu¬ 
sively  to  Marine  Corps  training  were  included.  Information  was  gathered  on 
jobs  identified  in  Department  of  Defense  Occupational  Conversion  Table  (1) 
categories  1,  2,  and  6— Electronic  Equipment  Repairmen,  Communication  and 
Intelligence  Specialists,  and  Electrical/Mechanical  Equipment  Repairmen. 

In  this  context,  the  phrase,  "Military  curriculum  development,"  refers  to 
the  "in-house"  development  and  revision  of  formal  military  courses.  No  inten¬ 
sive  effort  was  made  to  study  new  equipment  training  or  the  processes  by  which 
private  contractors  develop  initial  training  on  their  products. 

Particular  attention  was  paid  to  the  procedures  by  which  the  services 
gather  job  information  for  revising  their  training  curricula.  In  addition,  infor¬ 
mation  was  gathered  on  the  rate  of  change  (addition  and  deletion)  of  resident 
first-enlistment  technical  courses,  and  on  the  extent  to  which  the  services  pro¬ 
vide  training  career  fields. 


Approach  to  the  Study 


RATIONALE 

The  content  of  training  courses,  and  the  procedures  for  developing  this  con¬ 
tent  are  ultimately  measured  on  the  basis  of  the  quality  of  the  training  system 
output— that  is,  in  terms  of  the  job  capabilities  of  graduates.  An  alternative  to 
using  the  criterion  of  performance  on  the  job  is  that  of  comparing  the  procedures 
that  are  being  used  with  some  sort  of  standard  for  the  development  of  training 
content.  The  latter  approach  was  selected  for  this  study,  and  the  following  plan 
was  adopted: 

(1)  Develop  an  idealized  model  or  framework  to  represent  the  process 
or  set  of  procedures  to  be  used  in  developing  training,  basing  the  model  on  the 
cumulated  findings  and  practices  of  modern  training  research. 

(2)  Analyze  the  formal  procedures  and  practices  used  in  developing 
training  content  throughout  the  military  services  in  terms  of  the  idealized  model. 

This  approach  can  be  thought  of  as  process  in  contrast  to  product  analysis 
to  evaluate  a  system.  The  outcome  of  process  analysis  is  both  "stronger"  and 
"weaker"  than  the  outcome  of  product  analysis.  It  is  "weaker"  in  that  inferen¬ 
tial  steps  must  be  taken  between  what  one  is  ultimately  interested  in— the  prod¬ 
uct— and  the  subject  matter  used  in  the  analysis;  hence,  the  implications  developed 
out  of  the  study  cannot  be  applied  as  directly  to  the  product. 

It  is  also  "weaker"  in  that  results  of  process  analysis  are  quite  sensitive  to 
the  precise  way  in  which  the  process  is  conceived— that  is,  the  model  used.  More 
than  one  model  is  normally  applicable  to  any  complex  system,  and  the  choice  or 
development  of  "the  best"  is  subject  to  a  certain  degree  of  assessment  of  the 
state  of  the  art.  Thus,  it  is  important  to  recognize  that  the  conclusionj  and 
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implications  of  a  process  analysis— this  study  in  particular— rest  heavily  upon 
the  model  used.  The  model  should  be  viewed  as  a  set  of  assumptions  that  was 
not  tested  in  a  formal  way.  To  the  extent  the  assumptions  might  be  faulty,  the 
conclusions  and  implications  would  be  faulty;  to  the  extent  that  other  sets  of 
assumptions  might  have  been  '  equally  reasonable."  conclusions  and  implica¬ 
tions  derived  from  the  study  would  not  be  unique. 

The  "strength"  of  a  process  analysis  approach  rests  in  the  fact  that  it 
provides  specific  diagnostic  information  regarding  what  might  be  done  to  improve 
an  operational  system.  That  is,  while  product  analysis  provides  the  information 
that  "it  works"  or  "it  doesn't  work"  or  "it  works  to  thus  and  such  extent,"  it 
does  not  clearly  indicate  where  the  strengths  and  weaknesses  of  the  existing 
system  may  lie.  Process  analysis,  on  the  other  hand,  provides  direct  diagnos¬ 
tic  information  to  guide  specific  action. 

If  there  were  no  limit  on  time  or  resources  that  could  be  applied  to  a  study 
of  procedures  for  determining  the  content  of  technical  training  courses,  the 
most  desirable  solution  would  be  to  combine  product  and  process  analysis,  per¬ 
haps  including  several  alternate  models  to  form  the  framework  for  the  process 
analysis.  In  such  fashion,  one  would  have  the  ability  to  produce  direct  infor¬ 
mation  on  the  capabilities  of  the  product  and  also  detailed  data  toward  modifying 
the  process.  However,  under  the  constraints  of  time  and  resources  available 
for  this  study,  only  process  analysis  provided  a  feasible  approach  to  the  project. 

The  same  constraints  made  it  impractical  to  use  more  than  one  model  as 
the  basis  for  analysis.  However,  while  hypothetically  there  are  other  models 
that  might  have  been  used  to  represent  the  process  of  developing  a  training 
curriculum,  a  review  of  training  research  showed  there  is  substantial  consensus 
as  to  the  major  outlines  of  the  "ideal"  method  for  developing  training  content. 

No  alternate  models  that  were  essentially  different  than  the  one  employed  in  the 
analysis  were  identified. 

PROCEDURE 

The  first  step  for  the  study  consisted  of  devising  the  model  of  the  training 
development  process.  The  model  was  constructed  following  an  extensive  search 
of  the  training  and  training  research  literature.  The  literature  search  included 
a  study  of  the  list  of  reports  in  the  Defense  Documentation  Center  Technical 
Abstract  Bulletin  from  1958  to  1966.  Reports  that  were  selected  for  examina¬ 
tion  included  those  that  dealt  with  (a)  job  performance  evaluation  with  results 
pertinent  to  a  particular  course  of  military  instruction,  (b)  experimental  treat¬ 
ment  of  the  content  of  a  particular  course  of  military  instruction,  or  (c)  a  review 
of  training  research  literature. 

The  second  step  was  the  development  of  information  on  current  formal 
procedures  and  practices  in  the  Armed  Forces  in  order  to  analyze  these  proce¬ 
dures  and  practices  in  terms  of  the  idealized  model.  This  data  collection  step 
was  accomplished  by  visits  both  to  major  headquarters  concerned  with  training 
doctrine  and  formal  procedures,  and  to  field  installations  where  the  actual  con¬ 
duct  and  development  of  training  takes  place. 

Twenty-one  trips  were  made  to  locations  outside  the  Washington,  D.C.  area 
and  a  large  number  of  visits  and  telephone  contacts  were  made  \  ’thin  the  .irea. 
Military  installations  visited  included: 

Army 

Headquarters,  U.S.  Continental  Army  Command,  Fort  Monroe,  Va. 

U.S.  Army  Enlisted  Evaluation  Center,  Fort  Benjamin  Harrison,  Ind. 


U.S.  Army  Security  Agency  Training  Center  and  School,  Fort 
Devens,  Mass. 

U.S.  Army  Signal  Center  and  School,  Fort  Monmouth,  N.J. 

U  S.  Army  Transportation  School,  Fort  Eustis,  Va. 

Navy 

Headquarters,  U.S.  Naval  Air  Technical  Training  Command, 
Memphis,  Tenn. 

U  S.  Naval  Personnel  Research  Activity,  San  Diego,  Calif. 

U.S.  Naval  Examination  Center,  Great  Lakes,  HI. 

U.S  Naval  Training  Center,  Great  Lakes,  Ill. 

Pensacola  Naval  Station,  Pensacola,  Fla. 

Air  Force 

Training  Research  Division,  Behavioral  Sciences  Laboratory, 
Aeromedical  Research  Laboratory,  V/ right- Patterson  AFB,  Ohio 
Personnel  Research  Laboratory,  Aerospace  Medical  Division, 
Lackland  AFB,  Tex. 

Headquarters,  Air  Training  Command,  Randolph  AFB,  Tex. 
Operations  Branch,  Operations  Division,  Keesler  AFB,  Miss. 


Organization  of  tho  Report 

The  chapters  to  follow  deal  with  the  subject  matter  of  the  study  areas  as 
follows:  Chapter  2  presents  a  model  training  curriculum  development  process; 
Chapter  3  compares  the  curriculum  development  procedures  used  by  the  Army, 
Navy,  and  Air  Force  against  the  model;  Chapter  4  deals  with  the  last  step  in 
the  model,  that  of  feedback  from  job  performance  to  training;  Chapter  5  dis¬ 
cusses  problems  and  considerations  in  applying  the  model  in  the  services;  and 
Chapter  6  summarizes  the  major  findings. 


Chapter  2 

A  MODEL  FOR  TRAINING  CURRICULUM  DEVELOPMENT 


The  functioi.  of  military  training  curricula  is  to  produce  personnel  who  are 
capable  of  specifiable  performances  in  support  of  military  missions.  Modern 
training  research  has  generated  the  salient  features  of  a  model  for  military 
training  curriculum  development.  This  model  served  as  a  guide  to  the  efforts 
of  the  study  personnel,  and  as  a  set  of  criteria  against  which  to  compare  the 
existing  procedures  employed  by  the  military  services  in  developing  and  Mod¬ 
ifying  their  training  curricula. 


Steps  in  the  Development  of  a  Model  Training  Curriculum 

The  steps  of  the  training  curriculum  development  model  are: 

(1)  Conduct  system  analysis. 

(2)  Develop  task  inventory. 

(3)  Develop  job  model. 

(4)  Conduct  task  analysis. 

(5)  Derive  training  objectives- 

(6)  Develop  training  program- 

(7)  Monitor  trained  product  and  modify  training  curriculum  as  required. 
In  the  following  brief  description  of  these  steps,  a  number  of  references  are 

included  as  points  of  entry  to  the  larger  body  of  research  literature  bearing  on 
the  development  of  job-oriented  training  curricula. 


CONDUCT  SYSTEM  ANALYSIS 

The  ultimate  purpose  of  a  military  man- machine  system  is  mission  accom¬ 
plishment.  The  performance  requirements  of  a  particular  job  should  be  defined 
and  evaluated  relative  to  this  larger  frame.  The  first  step,  therefore,  is  to 
define  the  operational  system  of  which  the  job  is  a  part,  the  system's  missions 
and  goals,  the  functions  and  interactions  of  its  components,  and  the  environ¬ 
ments  in  which  it  operates  (Smith,  2), 

System  analysis  places  the  job  toward  which  training  is  to  be  designed  in 
the  perspective  of  the  mission  and  requirements  of  the  operational  system.  On 
the  basis  of  system  analysis,  the  importance  of  and  probable  gains  to  be  realised 
from  training  can  be  viewed  in  relation  to  other  system  factors,  such  as  logis¬ 
tics.  Current  field  performance  of  the  job  and  its  tasks  can  be  assessed  in 
terms  of  system  effectiveness.  Training  objectives  can  adequately  reflect  mis¬ 
sion  requirements.  In  short,  system  analysis  is  used  to  (a)  define  the  scope  of 
the  training  effort,  (b)  shape  its  design,  and  (c)  evaluate  the  remaining  steps  of 
the  model  development  process  in  terms  of  their  effects  on  the  efficiency  of 
the  system. 


Examples  of  research  bearing  on  system  analysis  are  Haggard  and  Lyons 
(3),  Me  Knight  and  Butler  (4),  Shriver  et  al.  (5),  and  Winick  et  al.  (6). 

Recent  developments  leading  toward  statements  of  standards  of  task  per¬ 
formance  derived  from  mission  requirements  are  described  by  Dunlap  and 
Associates  (7). 

DEVELOP  TASK  INVENTORIES 

Task  inventories  are  organized  lists  of  duties  and  tasks  designed  for  per¬ 
formance  by  personnel  in  the  system.  Individual  interviews  and  other  techniques 
can  aid  in  completing  task  inventories-  An  Air  Force  procedure  is  described  by 
Archer  and  Fruchter  (8),  and  its  information  yield  by  Morsh  (9).  Current  prac¬ 
tice  should  not  be  accepted  uncritically,  of  course.  Mission  profiles  and  other 
system  analysis  devices  can  aid  in  determining  the  system  effectiveness  of 
current  performances. 

Examples  of  research  on  development  of  task  inventories  are  found  in 
Heimstra  et  al.  (10,  11). 

DEVELOP  JOB  MODEL 

A  job  model  is  a  set  of  detailed  task  descriptions  defining  the  job  perform¬ 
ances  toward  which  training  is  to  b:  designed.  For  a  single  duty  position,  the 
model  comprises  the  tasks  of  that  position.  For  training  personnel  for  assign¬ 
ment  among  two  or  more  duty  positions  (as  in  training  for  an  Army  Military 
Occupational  Specialty,  Navy  rating.  Air  Force  Specialty,  or  other  family  of 
duty  positions),  the  job  model  must  be  a  composite  of  the  task  descriptions  for 
those  positions.  For  a  composite  job,  the  model  sh.  lid  present  criteria  used 
for  inclusion  or  exclusion  of  tasks- 

Many  training  research  efforts  have  made  impucit  use  of  the  concept  of  the 
job  model.  Explicit  use  of  the  concept  may  be  found  in  Ammerman  (12),  in 
Cogan  (13).  and  in  the  generation  of  job  types  by  Morsh  (14),  The  development 
of  detailed  task  descriptions  may  be  found  in  Brown  (15),  MacCasfTn  et  al.  (16). 
and  Woolman  (17). 

CONDUCT  TASK  ANALYSIS 

Each  of  the  tasks  included  and  described  in  detail  in  the  job  model  should 
be  analysed  to  determine  which  tasks  or  portions  of  tasks  should  be  allocated 
to  formal  training  and  also  to  provide  the  basis  on  which  precise,  job-oriented 
training  objectives  can  be  stated.  The  decision  to  devote  training  time  to  a  task 
should  be  made  in  terms  of  criteria  such  as  importance  to  mission  success, 
frequency  of  performance  on  the  job,  and  ease  of  learning  on  the  job. 

Smith  (3)  and  Chamberlain  {It)  present  alternative  systems  for  considering 
criteria  for  decisions  on  allocating  tasks  to  formal  training  or  to  on-thc-job 
learning.  It  is  doubtful  whether  service-wide  criteria  can  be  established  for 
allocating  content  to  formal  training  or  on-the-job  learning.  Guidance  on  the 
nature  of  applicable  criteria  should  be  furnished,  however,  and  each  curriculum 
should  contain  a  statement  of  the  criteria  used  for  allocation  in  its  case. 

In  general,  task  analysis  should  also  isolate  (a)  the  conditions  and  standards 
under  which  task  performance  occurs  in  the  operational  setting,  (b)  those  aspects 
of  a  task  the  performance  of  which  can  be  supported  by  job  aids,  thus  reducing 


training  requirements,  (c)  those  aspects  which  can  be  assumed  (or  demonstrated) 
to  be  already  learned  by  the  trainee,  and  (d)  those  aspects  which  must  be  learned, 
and  their  psychological  nature.  The  possibility  of  supporting  job  performance  by 
performance  aids,  thereby  reducing  training  requirements,  is  dealt  with  by 
Folley  and  Shettel  (19)  and  by  Topmiller  (20). 

Further  sources  of  information  on  task  analysis  are  Folley  (21),  Gustafson 
et  al.  (22),  Jones  and  Fairman  (23),  Legere  (24),  McKnight  and  Butler  (4), 

Naurath  and  Kelly  (25),  Shriver  (26),  Snyder  (27),  and  the  U-S.  Army  Quarter¬ 
master  School  (28). 

DERIVE  TRAINING  OBJECTIVES 

For  each  task  allocated  to  formal  training,  training  objectives  should  be 
derived  which  specify  the  performances  required  to  complete  the  task  to  an 
acceptable  level  of  proficiency.  Training  objectives  should  be  based  on  and 
responsive  to  the  findings  of  task  analysis. 

The  development  of  training  objectives  is  discussed  by  Ammerman  and 
Melching  (29).  Mager  (16),  Smith  (2),  and  the  U.S.  Army  Quartermaster  School 
(28).  Accepted  criteria  are  ths_t  an  objective  should  describe  a  job-relevant  per¬ 
formance.  the  conditions  under  which  it  is  to  be  observed  or  measured,  and  an 
associated  standard  defining  its  attainment. 

Research  on  the  development  of  training  objectives  is  reported  by  Ammer¬ 
man  (30),  Hoehn  (31).  and  Hoehn  and  McClure  (32). 

DEVELOP  TRAINING  PROGRAM 

Training  techniques  and  devices,  achievement  and  proficiency  tests,  and 
graduate  performance  specifications  may  all  be  unified  in  terms  of  precise!} 
stated  training  objectives-  The  generation  of  training  content  is  discussed  in 
Smith  (33).  Quality  control  of  training  content  is  discussed  in  Smith  (34),  and 
example  of  research  on  development  of  quality  control  measures  may  be  found 
in  Greer  et  al.  (35)  and  a  supplemental  report  by  Duffy  and  Colgan  (38).  * 

Training  methods,  the  topic  of  a  separate  CTEP  study  area,  will  not  be  dealt 
with  in  this  report. 

MONITOR  TRAINED  PRODUCT  AND  MODIFY  TRAINING 
CURRICULUM  AS  REQUIRED 

The  model  training  development  process  begins,  as  we  have  seen,  with  an 
analysis  of  the  system  and  the  job.  If  training  objectives  are  based  on  analysis 
of  the  tasks  of  the  job  and  if  these  same  training  objectives  dictate  the  content 
of  the  training  program,  it  would  seem  that  the  job  relevance  of  the  training  sub¬ 
ject  matter  is  assured.  It  is  important,  however,  to  stutfy  the  performance  of 
trained  graduates  in  the  llelo  in  order  to  (a)  verify  the  job-relevance  of  their 
training  in  terms  of  how  they  are  able  to  apply  »t,  ind  (hi  keep  training  content 
adaptive  to  changing  equipment  charnctenahm,  know-how,  and  field  conditions. 
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The  process  of  obtaining  information  on  graduate  performance  in  order  to 
insure  that  training  is  respons .  vt  to  the  needs  of  the  s>  stem,  often  referred  to  as 
"feedbar*',”  is  discussed  by  the  U  S  Army  Missile  and  Munitions  School  (37). 

It  is  the  topic  of  Chapter  4  of  this  report. 


Intsrrehtionihipt  Among  the  Results  of  Systems/ Job/Tosk  Analysis, 
f  reining  Curriculum  Content,  and  Graduate  Job  ferforesence 


The  performance  indicated  by  analysis  to  be  most  effective  for  system  mission 
accomplishment  (Designed),  those  taught  in  the  training  program  (Taught),  and  those 
used  on  the  job  (Used)  may  be  interrelated  as  shown  in  the  overlapping  circles. 


The  extent  of  agreement  between  performances  designed,  taught,  and  used 
may  have  implications  for  action  roughly  as  follows: 

(1)  Elimination  of  system*  ineffective  training  (Subarea  6),  system- 
ineffective  field  performance  (Subarea  7),  or  both  (Suban*  3). 

(2)  Introduction  of  system-effective  training  (Subarea  5)  into  both  the 
training  program  and  field  practice. 

(3)  Retention  of  system -effective  training  (Subareas  1  and  2). 

AU  areas  within  the  “Used*  circle  should  be  cirefuily  examined  during 
curriculum  design  and  in  subsequent  feedback  studies  for  possible  incorpora¬ 
tion  within  the  'Designed*  circle.  However,  the  fact  that  a  given  field  practice 
is  currently  employe*  does  not  necessarily  mean  that  it  is  sy stem- effective 

It  is  also  Uke'y  tha;  Jhe  optimum  interrelationship  between  the  circles  should  be: 


The  area  between  the  :wo  circles  above  corresponds  to  Subarea  4  in  the  pre¬ 
vious  set  of  circles 

Although  the  'Designed*  and  "Used*  areas  Mould  coincide  (ineffective  prac¬ 
tices  should  be  eliminated;  effective  field  practices  should  be  incorporated  into 
training  design),  die  content  of  the  training  curriculum  often  need  not  prepare 
the  trainee  tor  his  complete  job  performance.  That  is.  there  should  be  an  opti¬ 
mal  balance  between  what  content  is  included  in  the  training  program  and  what 
is  more  effectively  learned  on  the  job 

* 


Chapter  3 


TRAINING  CURRICULUM  DEVELOPMENT 
IN  THE  ARMY,  NAVY.  AND  AIR  FORCE 


At  stated  earlier,  the  major  objective  of  the  present  study  of  the  curriculum 
content  area  is  'to  identify  and  evaluate  current  procedures  for  determining  the 
content  of  technical  training  courses  in  terms  of  the  extent  to  which  these  pro* 
cedures  produce  courses  in  which  tt  e  content  adequately  supports  the  skill 
requirements  of  the  job  to  which  the  individual  is  assigned.' 

In  keeping  with  the  major  objective,  no  effort  has  been  made  to  ascertain 
whether  the  content  of  military  courses  of  instruction  ts,  in  particular  cases, 
job- relevant.  Instead,  the  procedures  suggested  by  training  research  for  insur¬ 
ing  the  job  relevance  of  training  content  (Chapter  2)  will  be  used  as  a  guide  for 
assessing  the  procedures  that  are  used  by  the  services.  The  presumption  may 
be  made  that  training  content  is  not  likely  to  be  entirely  job- relevant  unless 
there  are  procedures  for  adequately  insuring  this  t*e!evance. 

The  training  agencies  of  primary  concern  in  this  chapter  of  the  report  are 
(a)  for  the  Army,  the  U  S  Continental  Army  Command  (USCONARC)  and  14  of 
its  schools1  (those  engaged  in  training  for  DoD  Occupational  Conversion  Table 
categories  1.  2.  and  6);  (b)  for  the  Na-cy,  Bureau  of  Naval  Personnel  (NAVPERS 
or  BuPers)  and  Naval  Air  Technical  Training  Command  (NATECHTRACOM); 

(c)  for  the  Air  Force.  Air  Training  Command  (ATO  and  six  of  its  training  cen¬ 
ters.  those  engaged  in  training  for  Oof)  Conversion  Table  categories  1.  2.  and 

Conduct  lytha  Analysis 

No  indication  was  found  that  any  of  the  services  require  system  analysts 
for  training  purposes.  The  Air  Force  (and  perhaps  the  Army  and  Navy  as  well, 
although  this  was  mt  ascertained)  does  require  that  system  analysis  be  con¬ 
ducted  during  system  development  in  order  to  define  system  functions,  allocate 
functions  to  personnel  and  to  equipment,  determine  human  engineering  require¬ 
ments,  and  predict  manpower  requirements-  Provision  is  also  made  for  (ai  time¬ 
line  analysts,  the  analysis  of  a  sequential  list  of  system  functions  against  s  time 
base;  (b>  contingency  analysis,  the  analysts  of  nonroutine  functions  jurh  as 
equipment  malfunctions,  and  (c)  link  analysis,  analysis  of  the  frequency  and 
importance  of  interact  ions  between  system  elements  (3§> 

It  must  be  repeated  that  the  present  study  is  not  intensively  concerned  with 
the  processes  of  system  development  or  with  the  manner  tn  whieh  private  con- 
traetors  or  others  develop  training  curricula  prior  to  tne  time  the  system  is 
fielded— in  particular,  prior  to  the  time  military  curriculum  developers  assume 
responsibility  for  format  military  courses. 
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Although  the  Air  Force  requirements  for  training  information  and  plans 
generated  during  system  development  are  doubtless  of  valuable  significance  for 
military  curriculum  development,  no  indication  was  found  (a)  that  system  anal¬ 
yses  are  updated  during  the  life  of  the  system  after  its  fielding,  (b)  that  system 
analysis  is  viewed  as  the  first  step  in  military  curriculum  development,  or 
(c)  that  system  implications  are  formally  considered  during  curriculum  revision 


Develop  Tetk  Inventories 

The  Air  Force  and  the  Army  require  the  i>/elopment  of  task  inventories 
during  system  development;  the  Navy,  so  far  as  was  learned,  does  not.  No 
requirement  was  found  in  any  service  for  periodic  updating  of  task  inventories 
for  training  purposes  following  completion  of  system  development. 

The  Air  Force  and  the  Army  require  the  development  of  Qualitative  and 
Quantitative  Personnel  Requirements  Information  (QQPR1),  which  contain  task 
inventories.  QQPR1  is  updated  periodically  up  to  the  point  of  initial  system 
acquisition.  So  far  as  is  known,  however,  there  is  no  requirement  that  QQPR1 
be  maintained  and  made  available  to  training  personnel  during  the  life  of  the 
System  or,  more  relevantly,  that  it  be  periodically  updated  from  measures  taken 
on  current  job  activities. 

QQPRI  is  used  as  the  basis  for  Air  Force  Specialty  (AFS)  descriptions  con¬ 
tained  in  Air  Force  Manual  (AFM)  31-1,  Airman  Classification  Manual  (33)  and 
for  Military  Occupational  Specialty  (MOS)  descriptions  contained  in  Army  Regu¬ 
lation  (AR)  311-301,  Manual  of  Enlisted  Military  Occupational  Specialties  (40). 
Descriptions  in  these  manuals  include,  for  the  typical  AFS  or  MOS.  about  a  page 
of  general  duties  and  responsibilities  not  sufficiently  detailed  to  serve  as  a  task 
inventory.  Again,  no  requirement  was  found  for  this  information  to  be  periodi¬ 
cally  updated  from  data  collected  through  occupational  surveys  (of  the  type  des¬ 
cribed  in  AFM  33-3,  Occupational  Analysis  (41).  for  example). 

Air  Force  procedures  for  collecting  job  information,  aa  described  in 
AFM  33-3.  appear  well  suited  as  a  means  for  developing  task  inventories  (and 
for  providing  information  useful  in  constructing  job  models  ss  well).  As  stated 
in  the  manual,  information  generated  can  be  used  to  improve  the  accuracy  and 
completeness  of  specialty  descriptions,  maintain  currency  of  job  training  stand¬ 
ards,  determine  job  differences  and  relationships,  and  support  work  simplifica¬ 
tion  and  organisational  analysis  programs 

Detailed  procedural  guidance  for  collecting  comparable  information  in  the 
other  services  appeared  to  be  lacking.  None  of  the  services  provided  guidance 
for  analysing  syllable  job  information  for  the  purpose  of  developing  or  revising 
training  curricula.  Particularly  evident  was  the  absence  of  procedures  coordi¬ 
nating  the  efforts  of  the  several  agencies  engaged  in  various  aspects  of  operation 
s«3  research  in  curriculum  development,  system  development,  job  structuring, 
and  manpower  studies 


Develop  fob  Medal 

None  of  the  services  develop  adequate  job  model#  for  curriculum  devel¬ 
opment  purposes  Aa  stated  previously,  s  job  model  should  contain  detailed 
description*  of  the  job  performance*  (tasks}  toward  which  training  is  to 
be  designed. 


it 


Criteria  are  necessary  for  determining  whether  each  tsrk  in  the  tank  inven¬ 
tory  should  or  should  not  be  included  in  the  Job  model.  For  a  single  duty  position* 
for  -temple,  certain  tasks  may  be  performed  by  only  a  negligible  number  of  the 
job  incumbents;  such  tasks  should  be  excluded  from  the  model  for  failure  to 
meet  the  criterion  of  u  -quency  of  performance  by  incumbents-  Training  pro¬ 
grams  for  first-enlistment  personnel,  however,  are  seldom  developed  with 
respect  to  a  single  duty  position.  That  is,  most  often  graduates  are  expected  to 
be  assignable  to  one  or  another  of  two  or  more  duty  positions  within  an  AFS, 

MOS.  or  rating.  For  such  cases,  the  job  model  must  be  that  of  a  composite  job. 
and  criteria  for  including  or  excluding  tasks  should  include,  in  addition  to  fre¬ 
quency  of  performance,  generaHzability  of  learned  performance  from  the  task 
trained  to  other  similar  tasks  in  the  operational  setting. 

The  services  provide  neither  adequate  job  models,  nor  criteria,  nor  pro¬ 
visions  for  updating  job  information  for  training  purposes- 

The  statements  intended  to  serve  as  job  models  which  are  employed  by  the 
Army,  Navy,  and  Air  Force  stem  from  AR  611-201,  Manual  of  Enlisted  Military 
Occupational  Specialties  (40);  NAVPERS  16068B,  Manual  of  Qualifications  for 
Advancement  in  Rating  (42);  and  Air  Force  Regulation  50-94,  Job  Training 
Standards  (43). 

Army.  For  a  given  Military  Occupational  Specialty  (MOS).  the  job  model 
information  is  contained  in  the  appropriate  part  of  AR  611-201  under  -Duties 
and  Skills  and  Knowledges  •  *  Concerning  duties,  the  AR  states,  -This  section  of 
the  specification  provides  a  brief  statement  of  the  tasks  appropriate  to  the  spe¬ 
cific  MOS  without  regard  to  level  of  ski'l.*  Regarding  skilla  and  knowledges, 
the  AR  states.  ’This  section  of  the  specification  describes  skill  or  specific 
proficiency  level  within  each  MOS.  It  provides  a  guide  to  commanders  and  train¬ 
ing  agencies  and  assists  in  the  classification  of  positions  and  personnel  * 

Once  the  system  is  fielded,  the  MOS  specifications  are  the  responsibility 
of  the  Enlisted  Personnel  Branch.  Standards  and  Systems  Office.  Office  of 
Personnel  Operations-  In  that  office  nine  analysts,  each  responsible  for  one  or 
more  of  the  10  career  groups  which  encompass  about  450  MOSa,  monitor  con¬ 
tinuing  changes  in  MOS  specification*,  standards  of  grade  authorisation,  feeder 
patterns,  and  related  matters  Their  work  is  conducted  by  means  of  telephone 
conferences,  correspondence,  and  visits  to  and  from  field  agencies-  They  are 
neither  tuffed  nor  funded  u.  «nake  on- job  observations 

The  statements  <n  AR  611-2M  are  general  statements,  not  sufficiently 
precise  for  Uie  purposes  of  so  adequate  ;ofe  model  Tasks  are  not  included  or 
eluded  on  the  baaia  of  stated  criteria,  and  the  updating  process  is  cumbersome 
Navy  Navy  job  model  information  is  contained  in  NAVPERS  160668* 
referred  to  as  the  ’qoals  *  The  qoalt  are  intended  to  ’serve  as  s  gutdo  for  the 
preparation  of  training  course*,  training  publications,  on-the-job  training  pro¬ 
grams.  and  school  curricula  *  Also,  ’qualifications  do  not  prescribe  work 
requirements.*  and  ’because  they  are  minimum  requirement#*  qualification*  do 
not  cover  all  content  of  a  rating  ’ 

The  qoals  are  divided  into  alphabetically  designated  subject  matter 
trees  Each  area  includes  two  types  of  statement;  Practical  Factors*  described 
as  best  judged  by  actual  performance;  and  Knowledge  Factors,  beat  tested  by 
written  examination. 

Qoals  are  reviewed  on  a  three-year  cycle-  Technical  authorities,  pub¬ 
lications.  *>vw  equipment  data,  training  courses,  sad  training  personnel  are 
consulted  for  current  information.  From  this  information  a  questionnaire  is 
constructed  for  the  rating  (career  field).  The  questionnaire  is  mailed  to  a 
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maximum  of  10%  of  the  job  incumbents  in  the  rating.  Returns  (average,  about 
85%)  are  processed  by  computer  to  yield  summaries  by  installation  and  by  total. 
If  an  item  is  responded  to  positively  by  50%  or  more  of  the  respondents,  it  is 
accepted  as  a  qual.  The  development  and  processing  of  the  questionnaire  is 
handled  by  11  job  analysts  for  the  86  career  fields.  They  are  authorized  to  make 
shipboard  visits  to  observe  on-job  performance,  but  such  visits  are  few. 

The  quals  cannot  be  viewed  as  a  set  of  detailed  descriptions  of  job  per¬ 
formances  and  so  cannot  constitute  a  precise  statement  for  training  develop¬ 
ment  purposes. 

Air  Force.  The  Air  Force  has  a  document  approaching  the  job  model  in 
the  Job  Training  Standard  (JTS).  The  JTS  is  a  USAF -controlled  document 
(AFR  50-34),  responsibility  for  which  is  delegated  to  the  Air  Training  Command 
(A.CR  52-5).  Each  JTS  describes  an  airman  specialty  identified  in  AFM  39-1 
in  terms  of  general  knowledges  and  skills  with  proficiency  requirements  at  each 
AFS  skill  level.  JTSs  are  the  keystone  to  Air  Force  training.  They  constitute 
the  prime  source  documents  for  developing  instruction  (the  POI),  criterion 
measures,  on-the-job  training,  and  student  study  guides  and  workbooks.  JTSs 
are  developed  from  QQPR1,  AFM  39-1,  and  using  command  requirements;  they 
are  maintained  current  with  AFM  39-1,  operational  requirements,  permanent 
changes  in  training  capabilities,  and  the  latest  applicable  USAF  and  DoD  publi¬ 
cations  (ATCR  52-5). 

The  limitations  of  the  JTS  as  a  job  model  derive  from  its  orientation 
and  methods  for  its  maintenance.  ATCR  52-5,  Job  Training  Standards  (44), 
states  that,  "The  general  tasks  and  knowledges  listed  in  JTSs  are  Air  Force 
Specialty- oriented  to  reflect  the  consensus  of  major  air  command  requirements. 
They  are  not  job-oriented  .  .  (para  2a).  That  is,  JTSs  are  normally  prepared 
to  encompass  the  3,  5,  and  7  levels  in  an  AFSC  career  ladder  and  must  remain 
general  on  that  account  in  order  to  avoid  becoming  long  and  ineffective.  JTSs 
are  oriented  toward  career  progressions  and  not  specific  jobs.  Further,  no 
requirement  was  found  for  JTSs  to  be  periodically  reviewed  on  the  basis  of 
information  detailed  to  the  level  of  a  task  inventory,  although  provision  and  pro¬ 
cedures  for  collecting  this  information  are  contained  in  AFM  35-2. 

Summary  and  Comment.  The  Army  MOS  specifications  and  the  Navy  quals 
both  are  intended  to  serve  as  guides  for  training  development.  Both  are  also  used 
for  job  structuring  purposes.  The  Air  Force  JTS  has  only  one  purpose,  to  spec¬ 
ify  training  content.  All,  however,  are  expressed  in  sufficiently  gross  terms  that 
they  permit  wide  latitude  in  interpretation.  In  terms  of  the  model  training  devel¬ 
opment  presented  in  this  report,  none  can  be  described  as  a  precise  job  model. 


Conduct  Talk  Analyiii 

Once  system  analysis  has  been  completed,  task  inventories  have  been 
developed,  and  a  detailed  job  model  has  been  constructed,  task  analysis  should 
beg'.iiv  The  tasks  that  have  been  selected  on  carefully  weighted  criteria  and 
thoroughly  described  in  the  job  model  should  now  be  analyzed.  The  analysis 
should  have  the  twin  purposes  of  providing  sound  bases  for  (a)  decisions  on 
whether  to  allocate  tasks  to  formal  training  or  to  on-the-job  learning,  and  (b)  the 
derivation  of  training  objectives. 

U  is  important  to  distinguish  between  task  analysis  and  the  kind  of  job  anal¬ 
ysis  conducted  for  the  purpose  of  job  structuring.  The  MOS  structure  of 
the  Army  is  an  example  of  a  job  structure.  It  provides  for  10  occupational  areas 


encompassing  90-odd  career  groups;  standards  of  grade  authorization  are  set 
for  each  job  contained  in  the  structure.  Within  such  a  structure,  in  order  to 
provide  comparable  recompense  for  comparable  skill  and  responsibility,  it  is 
important  to  conduct  job  analyses.  But  job  analyses  for  toe  purpose  of  assess¬ 
ing  occupations  against  one  another  are  of  a  different  type  than  those  conducted 
for  training  purposes.  The  fact  that  a  job  has  been  analyzed  for  job -structure 
purposes  does  not  mean  that  the  analysis  will  be  sufficient  for  training  design. 

An  analysis  of  a  job  for  job  structure  purposes  may  be  accomplished  in  a  matter 
of  days  or  a  week;  the  initial  analysis  of  the  same  job  for  a  new  training  curric¬ 
ulum  may  well  require  months. 

So  far  as  was  discovered  in  this  study,  none  of  the  services  provide  a  require¬ 
ment  or  guidance  for  task  analysis  for  curriculum  development  purposes.  Nor 
are  criteria  provided  for  allocation  of  subject  matter  to  formal  training  or  to 
on-job  learning. 

Air  Force  documents  make  reference  to  task  analysis,  but  not  in  the  context 
nor  for  the  purpose  of  "in-house"  curriculum  development.  Thus,  AFSCM  80-3, 
Handbook  of  Instructions  for  Aerospace  Personnel  Subsystems  Design  (38)  states 
that  task  analyses  that  are  conducted  during  system  development  are  to  be  con¬ 
ducted  to  the  level  of  detail  specified  by  the  Personnel  Subsystem  Manager. 
These  analyses  arc  conducted  primarily  for  human  engineering  purposes,  allo¬ 
cation  of  functions  to  men  and  to  equipment,  and  manpower  projections.  They 
are  not  updated  after  the  completion  cf  system  development.  They  are  not  con¬ 
ducted  on  the  tasks  of  a  job  model  and  their  information  yield  is  not  directly  and 
adequately  applicable  to  military  curriculum  development  purposes.  No  Air 
Force  document  was  found  giving  guidance  on  task  analysis.’ 

No  Army- wide  requirement  for  task  analysis  was  found.  One  of  14  Conti¬ 
nental  Army  Command  schools  contacted  in  the  course  of  the  study,  the  U-S- 
Army  Quartermaster  School,  furnished  a  document  (28)  providing  guidance  on 
task  analysis.  Six  others  provided  publications  that  in  one  way  or  another  indi¬ 
cated  the  desirability  of  obtaining  job  information. 

One  Navy  document,  ED&TNGINTINST  5600.2  (45),  gives  as  the  first  step  in 
developing  or  revising  curricula,  "Analyze  the  jobs  and  duties  in  which  the 
graduates  of  the  course  will  perform,"  but  no  guidance  is  given  on  how  this  step 
is  to  be  accomplished,  nor  is  there  any  assurance  that  the  kind  of  analysis 
referred  to  is  task  analysis.  Neither  OPNAV  Instruction  3910. 4B  (46),  Technical 
Development  P^an,  Section  13,  Personnel  and  Training,  nor  NAVPERS  92684A 
(47),  Guide  for  Curriculum  Development,  refer  to  the  need  for  task  analysis.  A 
proposed  new  NAVPERS  Guide  for  the  Development  of  a  Curriculum  and  Train¬ 
ing  Materials  states:  "The  first  stage  in  the  development  of  a  course  of  instruc¬ 
tion  is  the  preparation  of  learning  objectives. "  Thus,  although  this  proposed 
new  Guide  commendably  emphasizes  the  importance  of  stating  learning  objec¬ 
tives,  it  carries  the  implication  that  the  objectives  so  prepared  will  not  be  firmly 
based  on  job  performance  requirements. 

As  for  allocation  of  subject  matter  to  formal  training  or  to  on-job  learning, 
no  Navy  documents  were  encountered  reflecting  concern  or  criteria  for  allocat¬ 
ing  content  to  resident,  port,  or  fleet  locations.  Allocations  to  resident,  field, 
or  on-job  locations  by  the  Air  Force  is  based  on  resident  training  capability. 

U  S.  Continental  Army  Command  policy  is  to  conduct  only  that  resident  instruc¬ 
tion  that  cannot  be  feasibly  administered  on  the  job  or  outside  the  school  system. 
Of  14  USCONARC  schools,  four  provide  local  guidance  documents  which  indicate 
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an  awareness  of  the  problem  of  where  to  allocate  the  conduct  of  training.  Two 
of  the  schools,  the  U.S.  Army  Missile  and  Munitions  Center  and  School  (formerly 
Ordnance  Guided  Missile  School)  and  the  U.S.  Army  Quartermaster  School,  pro 
vided  sets  of  criteria  for  allocating  subject  matter  to  formal  trai  ling.  Except  for 
these  two  Army  schools,  none  of  the  sour  es  in  the  study  provided  criteria  to  be 
used  in  deciding  whether  to  allocate  training  content  to  formal  instruction  or  to 
on-job  learning. 


Derive  ’reining  Objectives 

On  the  basis  of  the  information  made  available  tor  thin  report,  ft  has  been 
shown  that,  at  this  point  in  the  progression  through  the  steps  of  the  model  devel¬ 
opment,  no  service  provides  requirements  or  guidance  adequate  for  a  precise 
definition  of  the  job  toward  which  training  should  be  directed. 

Training  objectives  are  the  immediate  criterion  for  the  content  of  a  train¬ 
ing  curriculum.  Indeed,  they  are  designed  to  dictate  that  content.  Unless  the 
previous  steps  in  the  model  development  have  been  completed,  one  would  be 
unlikely  to  find  that  all  the  objectives  of  a  given  program  were  responsive  to  the 
needs  of  the  system,  to  the  job,  or  to  criteria  for  allocating  subject  matter  to 
formal  training. 

The  establishment  of  training  objectives  directly  responsive  to  the  results 
of  task  analysis  provides  a  firm,  job-oriented  basis  for  the  development  of  the 
training  program,  including  lesson  plans,  texts,  training  films,  devices,  practi¬ 
cal  exercises,  and  all  other  training  materials.  In  revising  curricula,  all  sub¬ 
ject  matter  not  directly  related  to  achievement  of  the  training  objectives  should 
be  pruned. 

A  clear  set  of  objectives  can  contribute  to  trainee  motivation.  There  need 
be  no  ambiguity  about  what  the  trainee  is  expected  to  learn,  and  his  efforts  can 
be  effectively  directed  toward  the  learning  goals. 

Precise  training  objectives  provide  clear  justification  for  training  facilities, 
course  length,  and  other  matters  in  the  administration  of  training.  They  also 
provide  a  sound  basis  for  a  quality  control  system  (achievement  and  proficiency 
testing)  for  continuous  monitoring  of  training  adequacy. 

In  spite  of  the  fact  that  there  are  no  service  directives  nor  guidance  for  pro¬ 
viding  the  curriculum  developer  with  a  precise  description  of  the  job,  all  services 
make  reference  to  the  need  for  job-relevance  in  stating  training  objectives. 

Army.  AR  350-1,  Army  Training  (48),  states  that  Army  subjects  and  pro¬ 
grams  of  instruction  are  to  be  based  on  an  analysis  of  the  knowledge  and  skills 
required  for  each  occupational  specialty.  U  S-  Continental  Army  Command  CON 
Reg  310-16,  Preparation  and  Processing  of  MOS  Army  Subject  Schedules  (49), 
requires  curriculum  developers  to  cite  one  or  more  learning  objectives  for  each 
period  of  instruction  under  the  overall  objective  of  qualifying  a  soldier  in  the 
grade  of  private  to  perform  the  duties  of  his  MOS  in  a  unit  engaged  in  or  support¬ 
ing  combat  operations.  It  states,  "A  learning  objective  will  state  what  the  soldier 
will  be  able  to  do  at  the  end  of  the  period  of  instruction.  It  will  not  state  what 
the  lesson  is  intended  to  do."  Also,  "Learning  objectives  will  state  the  task  the 
trainee  is  expected  to  perform  and,  if  appropriate,  the  conditions  under  which  he 
is  expected  to  perform  these  tasks,  and  the  standards  of  performance  expected." 
Also,  Annex  Q  to  CON  Reg  350-1  (50)  directs  the  use  of  objectives  relating  to 
general,  working,  and  qualified  knowledge. 

Of  the  14  USCONARC  schools  sampled  for  this  report,  seven  have  offi¬ 
cial  publications  that  mention  utilization  of  or  requirement  for  job  analysis  as  a 
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basis  for  the  development  of  training  objectives.  One  school,  U.S.  Army  Signal 
Center  and  School,  has  no  official  regulation  on  the  subject  but  an  experimental 
workshop  course  states,  "Valid  duty- oriented  objectives  can  be  derived  only  by 
means  of  a  thorough  job  analysis."  Another  school,  U.S.  Army  Quartermaster 
School,  has  a  guidance  document,  entitled  Duty-Oriented  Objectives  (28),  which 
provides  a  thorough  description  of  the  process  for  developing  training  objectives, 
from  task  analysis  to  determination  of  performance  standards. 

Navy.  Bureau  of  Naval  Personnel  ED&TNGINTINST  5600.2  (45)  requires 
personnel  developing  or  revising  curricula  to  select  and  write  general  and  spe¬ 
cific  learning  objectives  describing  in  short  statements  what  performance  is 
expected  from  the  trainee.  Also,  BuPers  Notice  1500,  Development  and  Imple¬ 
mentation  of  Learning  Objectives  (51),  requires  detailed  objectives  to  be  estab¬ 
lished  for  all  courses  under  Bureau  of  Naval  Personnel  supervision. 

NAVPERS  92566A,  Specifications  for  the  Preparation  of  Instructor's 
Guides  and  Trainee's  Guides  on  Naval  Equipment  (52),  states:  "The  objectives 
for  the  course  shall  be  prepared  first.  They  will  include  knowledge  and  skill 
levels  (on  specific  tasks)  to  be  attained  by  the  trainee."  The  Specifications  do 
not  describe  by  what  means  the  "specific  tasks"  are  derived. 

NAVPERS  93510,  Handbook  for  Writing  Learning  Objectives  (53),  states: 
"Items  of  terminal  behavior  are  normally  derived  from  those  knowledges  and 
skills  which  the  trainee  will  be  expected  to  use  after  graduation.  Behavior  des¬ 
cription  should  therefore  be  slanted  toward  the  jobs  or  tasks  the  graduate  will 
perform.  In  most  training  courses,  however,  job- slanted  behaviors  would 
require  topics  which  would  be  excessive  in  content  and  learning  objectives  which 
would  be  exceedingly  detailed."  [Underscoring  added]  This  statement  is  anti- 
theticalTo  the  principle  that  training  content  should  be  job- relevant. 

Naval  Air  Technical  Training  Command  CNATECHTRA  Instruction 
P1540.2C,  Training  Definitions  and  Specifications  (54),  states,  for  Naval  Air 
Technical  Training,  that  the  first  step  in  developing  a  curriculum  outline  is  to 
ascertain  and  prepare  the  objectives  of  the  course. 

In  summary,  of  the  Navy-wide  documents  encountered  in  this  study 
dealing  with  training  objectives,  the  only  one  that  makes  reference  to  the  need 
for  job- relevance  of  training  objectives  does  so  in  negative  terms.' 

Air  Force.  ATCR  52-18,  Management  of  Training  Materials  (56),  requires 
that  each  training  course  be  built  on  a  formal  specification  of  job  tasks  or  after¬ 
training  requirements  stated  in  behavioral  and  measurable  terms.  Also, 

ATCR  52-7,  Plan  of  Instruction  (Technical  Training)  (5_7),  identifies  the  key  unit 
of  a  POI  as  the  Statement  of  Learning  Objective  (SOLO),  defined  as  "the  identifi¬ 
cation  of  a  specific  observable/measurable  behavior  on  the  part  of  the  student, 
the  achievement  of  which  contributes  to  a  task  or  knowledge  specified  in  the 
training  standard"  [Underscoring  added].  Further  guidance  in  ATCR  52-7  on 
SOLOs  emphasizes  their  use  as  the  basis  for  instruction  nnd  testing,  and  requires 
them  to  be  clearly  worded  and  to  be  directly  reflective  of  the  training  standard. 

The  "training  standard"  referred  to  in  ATCR  52-7  is  the  Job  Training 
Standard  (JTS)  defined  in  ATCR  52-5  (44).  The  general  concept  of  a  Job  Train¬ 
ing  Standard  is  admirably  suited  to  training  curriculum  development.  However, 
its  present  effectiveness  is  diluted  by  the  lack  of  (a)  requirements  for  detailed 

'A  recent  yublic.it inn  (55,  Mtirch  1966)  of  the  I'.S.  Naval  Personnel  Research  Activity.  San  Diego. 
California,  contains  an  excellent  statement  on  training  objectives,  including  emphasis  on  the  need  for  job- 
relevance.  It  also  contains  guidance  on  training  curriculum  development,  using  a  number  ot  the  steps  in  the 
model  presented  in  this  report.  No  information  was  obtained,  however,  on  the  extent  to  which  the  document 
will  influence  Navy -wide  course  design,  as  do  the  above -referenced  Jocuments, 
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documents  bringing  system  and  task  data  to  bear  on  the  development  of  precise,  j 

detailed  job  models,  and  (b)  documented  evidence  of  systematic  analysis  of  the 
tasks  of  the  model  for  the  purpose  of  deriving  Air  Force  SOLOs.  The  JTS  in  its  j 

present  form  is,  as  described  in  AFR  50-34  (43),  a  document  containing  "general  j 

tasks,  knowledges,  and  proficiency  level  requirements."  It  is  intermediate 
between  a  job  description  and  a  listing  of  learning  objectives,  and  hence  serves  \ 

to  obscure  rather  than  delineate  a  precise  relationship  between  them.  i 

Discussion.  The  force  of  a  policy  statement  can  be  gauged,  in  rough  terms,  j 

by  its  requirements  for  documents  evidencing  compliance.  For  example,  as  j 

stated  above,  AR  350-1,  Army  Training  (48),  requires  that  Army  instruction  be  j 

based  on  an  analysis  of  the  knowledges  and  skills  required  for  the  corresponding 
occupational  specialties.  However,  the  only  Army-wide  document  defining  these 
knowledges  and  skills  is  AR  611-201,  the  Army  "MOS  book,"  which,  as  previously  1 

stated,  is  not  a  detailed  source  of  information  on  the  job.  Further,  there  is  no 
policy  requirement  for  documents  showing  the  progressive  distillation  of  system 
and  job  information  into  detailed  training  requirements. 

Requirements  are  levied  to  write  so-called  training  objectives  and 
guidance  is  provided  to  indicate  acceptable  wording  and  format.  However,  with¬ 
out  a  prior  requirement  to  collect,  document,  and  analyze  system  and  job  infor-  i 

mation,  the  writing  of  training  objectives  can  easily  reduce  to  a  paper  exercise. 

Until  documents  are  generated  detailing  the  correspondence  of  training 
content  with  system  needs,  the  temptation  to  describe  current  training  program 
content  in  different,  more  acceptable  words  will  continue.  Since  the  fifth  step  of 
the  model  development  process,  derivation  of  training  objectives,  requires  evi¬ 
dence  of  the  first  four  system-valid  products,  current  service  efforts  in  this 
direction  are  vitiated,  at  least  in  part. 

Admonitions  to  curriculum  developers  to  make  training  objectives  job¬ 
relevant  will  not  replace  directives  to  do  so,  complete  with  procedural  guidance 
for  complying.  Although  there  are  clear  instructions  available  to  curriculum 
developers  in  all  three  services  for  developing  well-worded  objectives,  there  is 
no  way  of  assuring  that  the  objectives  so  written  are  actually  responsive  to  the 
needs  of  the  system  or  relevant  to  the  job  performances  toward  which  training  is 
to  be  directed. 

Responsiveness  to  the  system  and  the  job  could  be  better  assured  by 
requirements  and  guidance  for  completing  the  earlier  steps  of  the  development 
model,  then  making  sure  that  training  objectives  are  justified  by  reference  to  the 
relevant  detailed  system  and  task  analysis  data. 


Develop  Training  Program 

The  content  of  a  training  curriculum,  except  for  certain  internal  pedagogic 
devices  such  as  orientations  and  reviews,  should  be  determined  by  its  training 
objectives.  The  training  methods  by  which  the  objectives  are  achieved  are  the 
topic  of  the  CTEP  Study  Area,  Training  Methods. 

Just  as  training  objectives  provide  a  precise  starting  point  and  definition  for 
the  development  of  detailed  training  content,  so  should  there  be  a  precise  speci¬ 
fication  of  the  trained  performance  capability  of  the  graduate  of  the  training 
program.  Such  specifications  are  needed  in  order  to  provide  for  detailed  feed¬ 
back  from  graduate  job  performance  to  the  corresponding  details  of  the  training 
program,  enabling  ready  revision  of  the  content  or  methods. 


The  Air  Force  Job  Training  Standard  is  the  nearest  approach  of  the  services 
to  a  set  of  graduate  performance  specifications.  As  was  previously  discussed, 
however,  it  is  inadequately  derived  from  system  needs  and  insufficiently  precise 
to  serve  as  an  ideal  set  of  specifications. 


Monitor  Trained  Product  end 
Modify  Training  Curriculum  a*  Required 

This  step,  which  deals  with  feedback  from  graduate  job  performance  to 
training,  is  covered  in  the  next  chapter. 


Chapter  4 

MONITORING  THE  TRAINEO  PRODUCT 


This  chapter  deals  with  the  last  step  in  the  model  training  curriculum  devel¬ 
opment  process.  The  full  title  of  that  step,  as  given  in  Chapter  2,  is  Monitor 
Trained  Product  and  Modify  Training  Curriculum  as  Required.  No  effort  has 
been  made  in  this  study  to  determine  the  extent  to  which  the  content  of  particular 
training  programs  is  modified  as  a  consequence  of  data  gathered  on  the  perform¬ 
ance  of  program  graduates.  Instead,  in  keeping  with  the  major  objective  of  the 
study,  attention  is  directed  to  the  procedures  for  obtaining  data  on  the  basis  of 
which  content  may  be  modified. 

The  process  of  obtaining  information  on  the  job  performance  of  the  trained 
graduate,  usually  referred  to  as  obtaining  feedback  from  the  job  to  training,  is 
nearly  as  important  as  the  steps  leading  to  the  development  of  the  training  pro¬ 
gram.  It  is  the  function  of  feedback  to  provide  verification  of  the  aaequacy  of 
the  trained  product  and  to  keep  curriculum  content  continuously  adjusted  to  the 
changing  system  and  job  conditions. 


Feedback  Effort*  in  the  Service* 


TECHNIQUES  USED  BY  THE  SERVICES 

Formal  techniques  used  by  the  services  to  obtain  feedback  from  job  to  course 
include  (a)  the  mailed  questionnaire  or  test;  <b)  the  field  visit,  in  which  inter¬ 
views  may  be  given  and  questionnaires  administered;  (c)  the  performance  evalu¬ 
ation,  a  study  of  graduate  trainee  iob  performance  over  an  extended  (usually 
three-month)  period  of  time;  and  (d)  debriefing  of  personnel  returning  from  the 
job  situation.  Informal  feedback  occurs  as  (a)  personnel  come  from  the  job  sit¬ 
uation  to  assume  instructional  and  other  training  duties,  and  (b)  field  commanders 
and  others  correspond  with  training  agencies. 

ARMY  FEEDBACK  EFFORTS 

For  the  purposes  of  this  study,  the  researchers  contacted  Continental  Army 
Command  and  14  of  its  schools,  those  offering  one  or  more  courses  in  DoD 
Occupational  Conversion  Table  (1)  categories  1,  2,  or  6  (see  Chapter  3  for  list 
of  schools). 

Guidance  from  U  S.  Continental  Army  Command.  Annex  Q  to  USCONARC 
Regulation  350-1  (50)  directs  school  commandants  to  use  feedback  data  from 
course  graduates  and  their  supervisors  to  evaluate  courses  (sec  Appendix  A). 
Annex  Q  states:  ‘Schools  may  send  questionnaires  to  graduates  and  their  imme¬ 
diate  supervisors  after  the  school -trained  skills  have  been  use  1  on  the  job  for 
approximately  3  to  6  months  to  obtain  feedback  data.  .  .  .Questionnaires  will 
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include,  but  will  not  be  limited  to,  items  designed  to  determine  the  difficulty  and 
the  frequ  ncy  of  specific  job  tasks  performed  by  the  graduates.  .  .  .  The  onsite 
observation  and  interrogation  of  school  graduates  and  their  supervisors  in  their 
work  assignments  is  the  most  valid  technique  for  obtaining  feedback  data. " 

USCONARC Schools'  Feedback  Efforts,  FY  1965.  Of  the  14  schools  sampled, 

12  furnished  questionnaires  designed  to  gather  information  from  graduate 
enlisted  personnel.  One  of  the  two  remaining  schools  furnished  officer  ques¬ 
tionnaires  but  none  for  enlisted  graduates;  the  other  school  furnished  no  ques¬ 
tionnaires,  but  stated  that  it  was  developing  a  comprehensive  questionnaire 
program.  None  of  the  schools  reported  having  conducted  interviews  with  grad¬ 
uates  during  FY  1965;  three  reported  having  made  field  trips  for  the  purpose  of 
gathering  feedback  information. 

Data  on  feedback  received  from  the  12  USCONARC  schools  using  ques¬ 
tionnaire*  for  enlisted  graduates  in  FY  1965  are  summarized  in  Table  1.  Some 
data  were  furnished  for  FY  1966  but  were  withheld  from  this  table  because  of 
their  incomplete  nature.  Table  1  shows  the  schools  represented  had  sampled 
over  80%  of  their  courses,  but  only  two  of  the  schools  showed  returns  over  70%. 

The  U  S-  Army  Missile  and  Munitions  Center  and  School's  Guidance  for  Field 
Feedback  Projects  (37)states:  "It  is  desirable  to  get  90  to  100%  returns  to  be 
assured  of  good  representation  of  the  group  sampled."  The  percent  returns 
from  supervisors  appear  to  be  no  higher  than  from  graduates- 

Content  of  USCONARC  Schools'  Questionnaires.  The  content  of  the  question¬ 
naires  for  graduates  fell  into  two  categories:  frequency  of  task  performance 
and  applicability  of  school  subject  matter. 

Concerning  frequency  of  task  performance,  four  schools  ask  the  gradu¬ 
ate  to  indicate  how  many  times  in  the  past  three  or  four  months  he  had  done  the 
task  (e.g..  never,  1-2  times,  3-5  times).  Two  schools  ask  whether  he  performs 
the  task  frequently,  occasionally,  or  never.  One  school  asks  him  to  describe  his 
current  duties  and  indicate  the  percent  of  his  time  spen,  on  each.  Three  of  the 
schools  also  ask  whether  the  graduate  rates  the  topic  as  easy  or  difficult- 

Concerning  applicability  of  school  subject  matter,  four  schools  list  sub¬ 
jects  from  the  curriculum  and  ask  the  graduate  what  emphasis  should  be  given 
to  them  or  whether  instructional  time  should  be  increased,  decreased,  or  kept 
the  same.  One  school  provides  no  list,  and  asks  the  graduate  to  list,  for  example, 
job  duties  the  course  did  not  train  him  to  perform. 

Nearly  all  the  questionnaires  for  supervisors  list  job  skills,  tasks,  or 
areas  of  the  graduate's  job  and  ask  the  supervisor  to  indicate  the  degree  of  pro¬ 
ficiency  shown  by  the  graduate  on  the  job.  One  school  asks  the  supervisor  to 
list  not  more  than  five  of  the  graduate's  major  duties  and  to  reply  to  questions 
such  ss.  "What  recommendations  do  you  have  for  improving  the  course  that 
would  better  equip  this  man  to  perform  in  the  job  for  which  he  wi?  trained  7  ‘ 
Another  school  asks  the  supervisor  to  indicate  whether  hours  devoted  to  subjects 
in  the  course  should  be  increased,  decreased,  or  left  the  same. 

Three  schools  furnished  questionnaires  for  commanders.  One  lists 
"training  objectives*  such  as,  "Prepares  salad  dressing,"  and  asks  the  commander 
to  indicate  whether  this  task  is  performed  better  by  school-trained  or  OdT  per¬ 
sonnel.  Another  asks  the  commander  to  rate  the  graduate  on  a  five-point  scale 
on  such  l  >pict  as,  "How  well  does  his  training  assist  him  in  overcoming  new 
and  different  situa'ions?"  The  remaining  school  asks  the  commander  to  evalu¬ 
ate  the  graduate  on,  for  example,  "inspecting,  servicing,  adjusting,  replacing, 
testing,  removing  and  installing  organisational  maintenance  level  parts,  assem¬ 
blies  and  components." 
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Informal  Feedback  in  the  Army.  A  U.S.  Army  Continental  Army  Command 
FactHfeeet,  dated  1  November  1965,  licta  10  informal  means  of  obtaining  feedback: 

(1)  “A  high  percentage  of  officers  recently  returned  from  overseas  and 
CONUS  command  and  staff  positions  re  assigned  to  the  staffs  and  faculties  of 
our  schools. 

(2)  *Hq  USCONARC  critically  reviews  every  new  and  revised  POI  and 
Army  Subject  Schedule.  These  are  not  adopted  until  the  Deputy  Chiefs  of  Staff 
have  formally  approved  the  adequacy  of  the  subject  matter  in  their  respective 
areas  of  interest. 

(3)  'Annual  commandants'  conferences  are  sponsored  by  the  Schools 
Directorate.  The  needs  of  the  field  units  are  main  points  of  discussion. 

(4)  'Practically  every  leadership  type  class  is  seasoned  with  a  sprin¬ 
kling  of  officers  recently  returned  from  overseas  assignments.  In  seminars* 
classroom  discussions,  and  in  their  end-of-course  critique  sheets,  comments  of 
these  students  serve  to  generate  changes- 

(5)  'Personal  letters  between  commanders  and  commandants. 

(6)  'In  some  instances,  schools  have  conducted  special  conferences  cf 
commanders  who  are  knowledgeable  in  the  schools'  instructional  areas  and  are 
recipients  of  the  schools'  trained  products. 

(1)  "Hq  USCONARC  personnel  routinely  visit  schools  and  CONUS  TOE 
units.  Indications  oi  inadequacies  of  school  training  are  fed  back  to  the  Schools 
Directorate  in  the  form  of  trip  reports  and  memoranda  for  record. 

(S>  -Instructor  conferences  and  workshops,  although  primarily  con¬ 
cerned  with  the  latest  ideas  on  improving  instructional  techniques,  also  deal  with 
the  practical  training  needs  of  the  field  soldier. 

(9)  -Periodic  Army  school  system  reviews  by  DA  Boards  of  Officers 
(Gerow.  Williams.  Daley.  Haines  Boards)  determine  adequacy  of  training. 

(10)  'Pragmatic  reports  such  as  Lessons  Learned  and  certain  HumRRO 
reports  feed  valuable  information  to  the  school  commandants-' 


NAVY  FEEDBACK  EFFORTS 

RuPert  has  no  forma)  procedures  for  feedback,  relying  primarily  on  the 
experience  of  personnel  rotated  back  from  the  field  fer  instructor  duly  and  to  a 
lesser  extent  on  complaints  from  the  field 

A  section  of  the  New  Developments  Branch,  Bureau  of  Naval  Personnel, 
called  Proficiency  .Measurement  and  Training  Feedback,  conducts  field  follow¬ 
ups  of  training  conducted  in  support  of  new  equipments-  At  present,  the  work  of 
this  group  »s  restricted  to  specialised.  C  school,  training,  and  studies  are  con¬ 
tacted  only  in  response  to  request-  There  is  no  directive  requiring  fleet  support. 
Reports  are  forwarded  »o  program  manager*  in  Pers  C.  and  to  the  proponent 
training  agency. 

XATECttTRACOM  employs  the  following  mean*  of  obtaining  feedback 
information: 

(1)  Rating  Task  Serve?*  and  Rating  Task  Survey  Report.  Personnel 
reporting  as  ft  school  students  or  C  school  instructors  are  regularly  required 
to  complete  a  Rating  Task  Survey.  The  Survey  contains  items  derived  from 
the  Manual  of  Qualification*  for  Advancement  in  Rating  (4|)  and  the  respond¬ 
ents  are  asked  to  indicate  the  frequenry  o'  performance  by  Class  A  graduates 
on  Uve  job.  The  data  are  summarised  and  evaluated  by  school  administrators. 

A  Rating  Task  Survey  Report,  baaed  on  at»«ut  1 90S  returns,  is  submitted 


» 


annually  to  the  CNATECHTRA.  There  it  no  requirement  for  action  beyond 
submission  of  the  report*  and  poasible  resulting  changes  are  not  monitored. 

(2)  Graduate  Evaluation  Report-  Six  months  after  the  A  school  gradu¬ 
ate's  arrival  at  his  duty  station,  his  supervisor  is  asked  to  complete  his  yellow 
card  and  return  it  to  NATECHTRACOM  Headquarters-  The  yellow  card  coiw'ns 
items  from  the  Manual  of  foalifications  for  Advancement  in  Rating  (42),  and  the 
supervisor  is  asked  to  rate  the  man  for  his  job  proficiency  on  these  items. 
Returns  run  about  10-12%.  Data  are  summarized  quarterly  and  sent  directly  to 
the  school  concerned  without  comment  or  action  required 

(3)  General  Aviation  Technical  Training  Conference.  There  is  a  bien¬ 
nial  three-day  conference  of  trainers  and  users  which  in  the  past  has  resulted 
in  a  number  of  actions  to  change  curricula. 

T«Ma  2 


Meaning  of  Air  feres  Training 
Evaluation  Division* 


AIR  FORCE  FEEDBACK  EFFORTS 


t*  tfcn*  ftwiy  helwtM  !)>>«»•>  m 
nvxwt  Vk**l  faitrt  twa,  in  m  iWw 


_ ^  _  Evaluation  of  individual  courses 

VtMnr  *i  Tn>wi*i  i  of  instruction  in  the  Air  Force  is 

-  r  I1'1."  '  '  _ "  conducted  in  accordance  with  AFR 

- - 1 - - J -  50-10  (5H)  This  regulation  pro- 

Amrilin*  2  t  9  vides  for  field  evaluation  visits. 

Ckttw*  l  2  :  lo  direct  correspondence,  question- 

K*c*l#r  *  -  :  s  nsires,  and  job  performance  evalu- 

UiLLmJ.  6  t>  ations.  as  defined  therein  These 

,  activities  are  carried  out.  in  the 

***•’  "  Air  Training  Command,  by  the  six 

Sktspwd  ilia  training  centera:  Amarillo,  Chanute 

T«—l  4  3  as  M  Keesler.  Lackland,  Lowry,  and 

*  uw-  UW4 1  .el—,  dm*.^  ~  Sheppard  W ithin  each  training  cen- 
ounivi  .*1  vi«*i  i,  m  tWw  ter ,  responsibility  for  conduct  of 

*,,i**iM  feedback  activities  is  gathered  in  s 

Training  Evaluation  Division  (TED) 

The  TEDs  are  manned  as  shown  in  Table  2  (administrative  personnel  not  included) 
The  TED  personnel  are  responsible  for  the  evaluation  of  all  training  given  at  the 
training  centers 

The  coverage  of  Airman  Basic  Resident  (ABR)  courses  (first  enlistment  tech¬ 
nical)  forecast  by  the  training  centers  in  FY  12*6  is  shown  in  TtbleS  R  is  impor¬ 
tant  to  note  that  about  5t%  of  all  ABR  courses  will  bsve  been  surveyed  by  mailed 
questionnaire,  44%  by  field  visits,  and  14%  by  job  performance  evaluations  Of  the 
ABR  courses,  f  4*  will  have  been  surveyed  by  at  least  one  of  these  methods  of 
obtaining  feedback  In  sll  methods,  the  Job  Training  Standard  is  oseu  as  the  guid¬ 
ing  criterion  of  training  effectiveness  The  methods  are  briefly  described  helow: 

Questionnaires’- Courses  awarding  an  AFSC  are  followed  up  m  1* -month 
cycles  A sample  I*  taken  over  a  three-month  per»*d.  question¬ 
naires  being  mailed  to  supervisors  about  it  days  after  the  man's 
grrduslion  Items  are  taken  from  the  related  Job  Training  Standard 
Ratings  are  unsatisfactory,  satisfactory,  outstanding,  or  not  performed 
Returns  run  close  to  ie%,  with  a  two-week  turnaround  for  COffUS 
installation*  Data  are  analysed  for  JTS  deficiencies,  utilisation  of 
JT5  tasks  and  knowledges,  equipments  being  maintained  or  operated 
rated  job  proficiency,  and  weighted  from  consideration  for  the  quali¬ 
fications  of  the  rater 


a* 


Field  Visits—  A  training  evaluation  officer  viaita  with  and  observes 
course  graduates  at  several  job  sites,  and  confers  with  their  super¬ 
visors.  concentrating  on  job  problems  amerable  to  a  training  solution 
Check  sheets  based  on  the  JTSa  are  available  but  seldom  used.  Field 
visits  are  often  used  to  follow  up  and  supplement  information  pre¬ 
viously  available  from  questionnaires- 
Job  Performance  Evaluations— Field  supervisors  rate  course  graduates 
daily  for  fo)  frequency  with  which  they  perform  items  listed  on  a 
check  sheet  (keyed  to  JTS  items),  (b)  adequacy  of  performance,  and 
(c)  how  much  relative  time  it  takes-  Reports  are  mailed  back  weekly 
for  10  weeks.  The  evaluation  begins  the  day  the  graduate  reports  for 
field  duty. 


TsUe  3 

Air  Force  Troining  Center  Evaluation  of  Fir$t-Eni«i'ment  Technical  Courses,  Ft  1946 
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FEEDBACK  EFFORTS  AT  THE  SECURITY  AGENCY  SCHOOL 

The  work  of  personnel  of  the  V  S  Army  Security  Agency  Training  Center  and 
School  (USASATCkS)  on  Development  at  Instructional  Systems  ft) IF,  requires 
•operate  consideration  ft  is  the  only  instance  encountered  during  this  study  of 
s  military  training  development  process  that  represents  sn  ongoing  application 
of  the  best  of  modern  training  development  technology  hi  the  OIS  process,  sys¬ 
tem  analyst*  is  conducted,  task  inventories  are  developed,  )ob  models  are  con¬ 
structed.  tasks  are  analysed  and  allocated  to  formal  training  -sr  on- job  learning, 
and  training  objectives  are  based  on  task  analysis  results. 

The  UN  ASA  Tv  fcS  effort  is  still  m  the  early  stages  One  ecwme  is  folly  oper¬ 
ational,  two  are  in  the  pilot  testing  stage,  and  all  are  expected  to  b*  operatic nal 
by  19?s  ft  is  thus  ton  early  to  describe  a  folly  operational  DIS  feedback  system 
The  initial  emphasis,  however,  is  on  field  vigils  by  trained  job  analysis  trams 
Questionnaires  will  not  be  used  unless  valid  and  adequate  information  can  be 
obtained  by  using  them 


An  interesting  Tact  is  that  DIS  personnel  plan  to  conduct  feedback  study  after 
the  graduate  has  been  on  the  job  only  one  month.  Usual  military  practice  calls 
for  graduate  job  tenure  of  three  to  six  months  or  more  before  the  gathering  of 
feedback  information.  DIS  personnel  reason  that  the  longer  feedback  gathering 
is  delayed,  the  more  difficult  it  becomes  to  determine  whether  graduate  perform¬ 
ance  is  the  result  of  formaltraining  or  to  what  extent  it  hasTieen  influenced  by 
on-job  learning.  Of  importance  here  is  the  fact  that  USASATC&S  requires  full 
job-qualification  of  its  trainees  at  graduation.  Most  other  systems  do  not  require 
such  a  high  standard  at  job- entry,  and  expect  larger  or  smaller  amounts  of  on- 
job  learning  to  occur.  Because  of  his  high  initial  capability,  the  USASATC&S 
graduate  may  perform  on  a  wider  sample  of  tasks  in  a  shorter  time  than  the 
typical  graduate. 

The  DIS  objective  is  that  feedback  studies  will  be  conducted  in  the  same 
manner  as  the  initial  job  analysis  for  a  new  course.  Thus,  the  trained  analysts 
wili  be  armed  not  merely  with  general  statements  such  as  those  in  the  Army 
MOS  Code  Descriptions,  the  Navy  "Quals,"  or  the  Air  Force  JTS,  but  with 
detailed  task  inventories,  task  descriptions,  task  analyses,  and  job-oriented 
training  objectives.  Analysts  are  also  provided  with  guidance  documents  such 
as  the  118-page  USASA  Command  Job  Analysis  System.  Job  Analysis  Handbook 
and  Guide  (24)  This  document  provides  guidance  for  the  preparation  of  a  Job 
Analysis  Report  on  the  field  command  organization,  mission,  and  work  activity 
analysis  for  the  particular  MOS;  identifying  information;  details  of  duties,  tasks, 
and  elements;  general  information;  and  equipment  lists. 


Studies  Conducted  to  Obtoin  Feedback 

The  following  summary  of  service  studies  to  gather  feedback  information 
stems  from  a  search  of  the  Technical  Abstract  Bulletins  of  the  Defense  Documen¬ 
tation  Center  from  1958  to  1966,  ineluding  several  training  bibliographies 
listed  therein. 

Army  Studies.  Very  few  reports  are  found  to  deal  with  feedback  from  the 
job  to  specific  Army  courses  of  technical  instruction.  Those  presenting  feed¬ 
back  information  were  not  primarily  designed  for  that  purpose.  The  studies  con¬ 
ducted  were  not  the  result  of  a  formal  evaluation  program  instituted  by  the  Army; 
they  were  studies  arising  from  research  and  development  activities  of,  for 
example,  HumRRO.  The  typical  study  involved  comparison  of  a  current  Army 
course  of  instruction  with  an  experimental  course  designed  on  the  basis  of  an 
analysis  of  the  job.  (Examples  of  such  studies  are  5,  16,  59,  60.  Studies  in 
which  graduates  were  tested  for  feedback  purposes  are  Cl  and  62.) 

Navy  Studies-  A  number  of  Navy  studies  were  found  dealing  to  a  greater  or 
lesser  extent  with  feedback.  Some  of  these  studies  consisted  of  shipboard  obser¬ 
vation,  interview,  and  test  of,  for  example,  Aviation  Structural  Mechanics  (63), 
Electronics  Technicians  (64-70),  Fire  Control  Technicians  (71,  72),  Shipfitters 
(73),  and  Sonarmen  (74-78).  These  studies  were  not  expressly  designed  to  eval¬ 
uate  specific  courses  of  instruction.  They  dealt  with  perr  nnel  in  particular 
ratings,  but  from  varied  course  backgrounds.  Thus,  a  typ.cal  report  (71)  states: 
"A  sample  of  332  first- enlistment  FT’s  from  74  ships  of  CruDesPac,  PhibPac, 
and  AirPac  was  contacted.  Most  of  these  men  had  Class  ’A'  and  Class  'C'  school 
training  and  were  either  FTA’s  or  FTM's." 

Several  Navy  studies  were  designed  to  evaluate  specific  courses  of 
instruction,  such  as  courses  for  Aviation  Electrician's  Mates  (79),  Aviation 


Electronics  Technicians  (80,  81),  Aviation  Machinist's  Mates  (82),  Aviation 
Structural  Mechanics  (83),  and  Torpedoman’s  Mates  (84).  These  studies  employed 
varied  means  of  evaluation: 

(1)  Tests.  In  some  of  the  studies  (80,  83,  84)  performance  tests  , 
were  administered  to  job  incumbents. 

(2)  Job  visit.  In  one  study  (79),  job  sites  were  visited  and  inter¬ 
views,  tests,  ratings,  and  job  diaries  were  administered. 

(3)  Performance  evaluation.  In  two  studies  (81,  82),  course  grad¬ 
uates  kept  job  diaries  for  a  period  of  12  weeks  on  the  job  following  graduation. 

Air  Force  Studies-  The  Air  Force  studies  on  feedback  from  the  job  to  indi¬ 
vidual  courses  provided  an  impressive  series  of  reports  •  oming  out  of  the  Air 
Training  Command  Project  Office,  Eglin  Air  Force  Base,  Florida  (85).  This 
series,  which  began  in  1953,  produced  114  reports  from  195''  to  its  termination 
in  1965.  It  averaged,  then,  about  15  per  year. 

The  procedure  used  in  the  Eglin  studies  (86)  was  to  pay  an  initial  visit 
to  a  job  site  where  recent  course  graduates  had  been  assigned.  The  site  was 
chosen  on  the  basis  of  an  ATC  priority  course  list.  Supervisors  of  the  graduates 
were  oriented  to  the  purposes  of  the  study  and  were  provided  forms  for  record¬ 
ing  the  graduate's  job  performance.  While  the  graduates  (from  3  to  25;  average 
10)  performed  normal  duties,  the  supervisors  completed  daily  and  weekly  sum¬ 
maries.  At  the  end  of  a  three- month  period,  a  terminal  visit  was  paid  and  grad¬ 
uates  and  supervisors  were  interviewed.  Air  Training  Command  required  the 
training  center  concerned  to  reply  as  to  what  action  was  being  taken  to  comply 
with  the  study  recommendations. 

In  the  roughly  eight-year  period  from  1958  to  the  end  of  FY  1965,  the 
Eglin  reports  evaluated  98  courses,  21  courses  more  than  once.  Since  the  pres¬ 
ent  number  of  Air  Force  active  ABR  (first- enlistment  technical)  courses  is  about 
150,  it  is  clear  that  the  Eglin  project  fell  considerably  short  of  providing  feed¬ 
back  for  all  Air  Force  technical  courses.  Yet,  considering  its  personnel  strength 
(1  Project  Officer,  2  GS-13  evaluators,  1  enlisted  man,  1  secretary),  the  output 
was  prodigious. 

The  guiding  criterion  of  graduate  job  performance  in  the  Eglin  studies 
was  the  Job  Training  Standard.  The  JTS  lists  the  required  knowledges  and  tasks 
for  the  job  and  indicates  required  proficiency  levels- 

The  Eglin  project  was  discontinued  in  June  1965.  Its  function  is  now 
being  carried  out  by  the  Evaluation  Divisions  of  the  Air  Force  Training  Centers. 

Utility  of  Data  From  Tosts  for  Advancement 

Proficiency  evaluation  measures  for  advancement  and  other  purposes  are 
developed  in  the  Army  by  the  Enlisted  Evaluation  Center,  Fort  Benjamin  Harri¬ 
son,  Indiana;  in  the  Navy,  by  the  Naval  Examining  Cente.1,  Great  Lakes  Naval 
Training  Center,  Illinois;  and  in  the  Air  Force,  at  the  6570th  Personnel  Research 
Laboratory,  Lackland  Air  Force  Base,  Texas. 

Army  Test'Procedures.  The  Enlisted  Evaluation  Center  (EEC)  collects  two 
types  of  data,  the  MOS  Evaluation  Tests  and  Commander  Evaluation  Reports 
(CER)  The  latter  consist  of  rating  forms  filled  out  by  both  the  supervisor  and 
the  supervisor's  supervisor.  These  forms  include  scales  for  rating  the  indi¬ 
vidual’s  cooperativeness,  reliability,  job  performance,  and  like  factors.  Items 
for  the  MOS  Evaluation  Tests  are  written  by  subject  matter  specialists  (usually 
instructors)  at  Army  service  schools,  following  outlines  constructed  by  EEC 
and  approved  by  the  schools. 
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After  tests  are  administered,  the  EEC  scores  the  CERs  and  Evaluation 
Tests,  and  derives  an  Evaluation  Score.  An  MOS  Evaluation  Data  Report  (EDR) 
is  prepared  for  each  individual  showing  his  score  and  relative  standing  in  each 
subject  area.  The  MOS  Evaluation  Score  and  EDR  are  used  as  the  basis  for: 

(1)  Awarding  Proficiency  Pay 

(2)  Verifying  ability  of  personnel  to  perform  duties  of  primary  and 
secondary  MOS. 

(3)  Qualifying  individuals  for  promotion. 

(4)  Determining  the  pay  grade  and  MOS  for  officers  reverting  to 
enlisted  status. 

(5)  Identifying  training  needs,  both  for  the  individual  and  his  unit. 
Navy  Test  Procedures.  Examinations  for  advancement  from  pay  grades 

E-2  through  E-7  are  developed  by  chief  petty  officers  permanently  assigned  to 
the  NEC.  Each  examination  is  designed  to  cover  advancement  qualifications 
prescribed  for  each  rate  and  rating  in  the  Manual  of  Qualifications  for  Advance¬ 
ment  in  Rating  (42).  These  qualifications,  or  "quals,"  constitute  the  basis  not 
only  for  advancement  tests,  but  also  for  a  sizable  share  of  curriculum  construc¬ 
tion-training  personnel  are,  understandably,  concerned  that  students  be  able  to 
pass  the  quals.  Items  for  advancement  tests  cover  subject  matter  listed  in  the 
annually  revised  bibliography,  Training  Publications  for  Advancement  in  Rating. 
Several  months  prior  to  the  examination,  each  candidate  is  sent  an  Examination 
Information  Sheet  showing  the  qualifications  for  his  rate/rating  and  a  bibliog¬ 
raphy  of  references. 

The  score  obtained  on  the  examination  is  the  principal  factor  in  advance¬ 
ment.  A  composite  score  is  obtained  from  the  following  factors: 

Examination  score  80  points 

Performance  factor  50  points 

(Commander's  rating) 

Length  of  service  20  points 

Time  in  grade  20  points 

Awards  10  points 

Maximum  composite  score  180  points 

Air  Force  Test  Procedures.  The  Air  Force  uses  three  specialty  knowledge 
tests  (SKTs),  with  few  exceptions  within  each  AFS,  for  advancement  and  promo¬ 
tion  within  each  of  the  3,  5,  and  7  skill  levels.  AFM  35-1,  Classification  Policy 
Manual  (87),  clearly  states  that  SKTs  are  measures  of  technical  knowledge 
required  for  award  of  an  AFSC,  and  do  not  measure  performance  on  the  job. 

Test  results  are  relative  and  serve  their  purpose  only  when  considered  along 
with  all  other  criteria  for  upgrading. 

SKT  items  are  constructed  by  subject  matter  specialists— senior  NCOs 
in  the  AFS— placed  on  temporary  duty  to  the  Personnel  Research  Laboratory. 

Advancement  Tests  as  Indicators  of  Training  Effectiveness.  Several  con¬ 
siderations  act  to  contaminate  the  value  of  written  advancement  tests  as  indices 
of  training  effectiveness. 

(1)  In  the  main,  examinations  are  based  on  occupational  descry,  aons, 
which  are  developed  by  personnel  agencies  for  purposes  of  utilization  and  assign¬ 
ment,  and  are  not  sufficiently  detailed  to  constitute  the  basis  for  job  (as  opposed 
to  occupational)  examinations. 

(2)  School/center  training  is  designed  to  equip  a  man  with  job-entry 
skills,  while  examination  for  upgrading  is  usually  conducted  on  completion  of  on- 
the-job  training.  Scores  thus  reflect  both  resident  and  on-job  training,  making 

it  difficult  to  separate  out  the  effects  of  each. 
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(3)  Advancement  tests  are  paper-and-pencil  measures,  and,  on  this 
account,  are  only  indirect  measures  of  job  proficiency. 

(4)  Tests  are  indicators  of  the  relative  standing  of  individuals  taking 
the  examination;  standards  are  based  on  the  scores  of  the  personnel  taking  the 
test  and  not  on  the  job  itself. 

Within  these  constraints,  however,  advancement  tests  may  be  treated  as 
trouble  indicators,  suggesting  deficiencies  to  be  checked  against  results  of 
further  study. 


Critique  of  Service  Feedback  Efforts 

All  the  services  are  concerned  with  the  problem  of  obtaining  feedback  from 
the  job  either  through  formal  programs,  studies,  or  informal  means.  With  the 
exception  of  the  Bureau  of  Naval  Personnel,  all  services  have  formal  programs 
for  eliciting  information  regarding  graduate  performance  on  the  job.  These 
programs,  however,  vary  widely  in  scope  and  content. 

The  g-eatest  divergence  in  feedback  efforts  is  between  the  plans  for  feed¬ 
back  studies  at  the  U  S  Army  Security  Agency  Training  Center  and  School  and 
the  efforts  conducted  in  the  rest  of  the  services.  The  major  difference  is  indi¬ 
cated  by  the  fact  t.  at  DIS  personnel  plan  to  obtain  feedback  using  the  methods  of 
initial  course  design.  The  rest  of  the  services  cannot  immediately  employ  this 
method  for  feedback  purposes  because  their  courses  were  not  designed  accord¬ 
ing  to  the  steps  of  the  curriculum  development  model  (Chapter  2)  as  are  the  DIS 
courses-  Complete  agreement  between  the  methods  of  course  design  and  those 
for  feedback  is  possible  in  the  DIS  approach  because  both  are  attuned  to  the 
system  and  the  job.  Part  of  the  reason  that  such  wide  variance  in  approach  to 
feedback  is  found  throughout  the  rest  of  the  services  is  no  doubt  attributable  to 
the  disconcerting  effects  of  the  attempt  to  feed  back  information  on  job  perform¬ 
ance  to  course  content  which  was  not  designed  completely  in  terms  of  that 
performance. 

The  most  prevalent  means  of  obtaining  feedback  information  is  the  question¬ 
naire.  The  method  used  to  collect  information  will  vary  in  effectiveness  with 
the  purpose.  An  Air  Force  study  (88)  suggests  that  the  questionnaire  technique 
may  be  adequate  for  spotting  gross  deficiencies  and  is  relatively  inexpensive, 
but  that  more  dependable  information  requires  personal  contacts.  The  Air 
Force  appears  unique  among  the  services  in  scheduling  fairly  routine  field  visits. 

Feedback  for  the  purpose  of  validating  training  content  presumes  a  precise 
definition  of  what  that  content  is.  This  definition  takes  the  form  of  a  set  of  per¬ 
formance  characteristics  to  be  expected  of  course  graduates  on  the  job.  The 
assessment  of  whether  an  individual  has  met  the  standard  cannot  be  made  until 
the  standard  is  made  public.  A  document  specifying  capabilities  to  be  expected 
at  the  job-entry  level  is  necessary  in  order  (a)  ihat  these  specifications  may  be 
checked  against  operational  requirements,  and  (b)  to  provide  a  base  reference 
for  field  evaluation 

Academic  summaries,  if  not  performance  specifications,  on  graduates  of 
Army  and  NATECHTRACOM  instruction  are  carried  in  the  individual's  person¬ 
nel  jacket.  However,  comparable  information  on  BuPers  students,  their  "hard 
cards,"  are  retained  at  the  training  site.  Of  the  services,  only  the  Air  Force 
provides  a  public  document  describing  job  capebilities  to  be  expected  of  course 
graduates,  in  the  Job  Training  Standard.  The  value  of  these  standards  for  feed¬ 
back  purposes  may  be  expected  to  vary  directly  with  their  precision  of  statement. 
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A  sound  feedback  system  should  also  include  provision  for  detecting  the 
efforts  of  overtraining,  as  well  as  undertraining.  There  exists  no  known  policy 
statement  specifying  this  objective  for  any  of  the  services.  Systems  which  rely 
heavily  on  comments  from  the  field  are  unlikely  to  provide  information  on  over¬ 
training,  since  field  commanders  are  more  prone  to  comment  on  deficiencies 
than  on  surpluses;  overtraining,  if  it  exists,  will  likely  remain  undetected  until 
methods  are  devised  specifically  suited  to  pick  it  up. 

In  order  to  obtain  the  clearest  assessment  of  resident  training  effects,  feed¬ 
back  data  should  be  collected  shortly  after  the  man  arrives  at  his  duty  station 
and  has  been  put  to  work.  Otherwise,  on-job  effects  may  be  difficult  to  separate 
out.  Under  present  procedures,  feedback  is  usually  solicited  only  after  a  delay 
of  five  to  six  months.  Exceptions  include  USA3ATC&S  plans  for  DIS  and  the 
recently  adopted  Job  Performance  Evaluation  technique  used  in  the  Air  Force. 
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Chapter  5 

TOWARD  AN  IMPROVED  TRAINING  POSTURE 


This  chapter  examines  (a)  the  status  of  training  as  a  service  career  field, 

(b)  current  courses  in  preparation  for  curriculum  development,  (c)  the  need  for 
dissemination  of  information  useful  in  curriculum  development,  (d)  information 
on  the  rate  of  change  (addition  and  deletion)  of  service  courses,  and  (e)  informa¬ 
tion  related  to  implementation  of  job- oriented  curricula. 

The  Status  of  Training  as  a  Service  Career  Field 

Since  World  War  II  the"e  has  been  an  enormous  development  of  the  technol¬ 
ogy  of  curriculum  development  There  has  consequently  been  a  growing  require¬ 
ment  for  professionalism  in  the  field  of  training.  Training  in  the  Army  and  Navy, 
and  to  a  lesser  extent  in  the  Air  Force,  is  considered  something  that  any  officer, 
NCO,  petty  officer,  or  senior  airman  can  do.  An  enormous  price  in  both  effec¬ 
tiveness  and  cost  is  being  paid  for  this  assumption. 

There  has  been  a  considerable  lag  in  the  adoption  of  the  results  of  training 
research  by  the  services  The  steps  of  the  training  development  model  have,  in 
the  main,  been  current  in  the  field  of  training  research  since  the  mid-1950s. 
Their  implementation,  and  other  advances  in  training  technology,  could  be 
hastened  by  providing  training  career  fields  in  proportion  to  the  importance  of 
training  as  a  military  subject  matter 

Military  efforts  are  not  necessarily  gauged  by  their  Defense  Budget  alloca¬ 
tions  Nevertheless,  the  training  area,  which  is  currently  accorded  about  6%  of 
the  budget,  is  supported  by  a  career  field  in  only  one  of  the  services;  and  in  that 
service  less  than  1%  of  its  total  personnel  strength  is  assigned  to  a  training 
career  field 

Army.  The  Army  MOS  structure  provides  no  career  group  for  training 
either  for  officers  or  for  enlisted  personnel.  In  its  classification  code,  the  pre¬ 
fix  digit  8  identifies  officers  with  instructor  experiences;  for  enlisted  personnel, 
the  letter  H  in  the  fifth  alphanumeric  position  denotes  Instructor,  B  denotes 
Drill  Sergeant  (basic  training  instructor). 

Navy.  The  Navy  does  not  have  a  career  area  for  training.  Naval  officer 
career  areas  provide  for  sub-specializations,  identified  by  Naval  Officer  Billet 
Classification  (NOBC)  codes,  of  which  34  relate  in  some  manner  to  training 
activities.  However,  it  is  not  intended  that  an  officer  devote  his  career  to  an 
area  denoted  by  an  NOBC. 

There  is  no  Naval  enlisted  rating  for  training.  The  Navy's  Manual  of 
Qualifications  for  Advancement  in  Rating  (42)  under  "Military  Requirements  for 
Petty  Officers"  provides  a  number  of  Practical  and  Knowledge  Factors  on 
which  all  Naval  enlisted  personnel  must  demonstrate  proficiency  to  qualify  for 
advancement.  The  factors  stated  make  no  reference  to  any  of  the  elements 
recommended  in  this  report  for  curriculum  development. 
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Air  Force.  The  Air  Force  is  unique  among  the  services  in  providing  a 
career  area  for  training.  The  Air  Force  classification  structure  provides  15 
Officer  Career  Areas  containing  48  Utilization  Fields,  and  46  Airman  (enlisted) 
Career  Fields.  The  Officer  Air  Force  Specialty  Codes  (AFSCs)  are  found  within 
the  Personnel  Resources  Management  Career  Area  in  the  Education  and  Training 
Utilization  Field.  The  enlisted  AFSCs  for  training  are  found  within  the  Airman 
Education  and  Training  Utilization  Field  (officer)  and  the  Education  and  Train¬ 
ing  Career  Field  (enlisted),  together  with  authorized  and  assigned  strengths. 

The  total  officer  and  enlisted  personnel  assigned  in  the  Education  and 
Training  Utilization  and  Career  Fields  (shown  in  Table  4)  represents  about  0.7 
percent  of  the  total  current  Air  Force  strength. 

Table  4 


Air  Force  Education  and  Training  Utilization 
and  Career  Field  Strengths 


Af’SC 

Title 

Current 
j  Authorized 

Assigned 

Officers* 

7511  b 

Education  and  Training  Staff  Officer 

0 

296 

7?  If. 

Education  and  Training  Staff  Officer 

770 

513 

7521 b 

Education  and  Training  Officer 

0 

'll 

7524 

Education  and  Training  Officer 

332 

216 

7531 b 

Instructor 

0 

111 

7535 

Instructor 

1093 

839 

Total 

2195 

2302 

Enlisted 

Personnel' 

75190 

Education  and  Training  Supervisor 

107 

166 

75170 

e  ducation  Supervisor 

154 

172 

75171 

Audio- \  isual  Technician 

53 

84 

75172 

Training  Technician 

1464 

1391 

75150 

Education  Specialist 

245 

131 

75151 

Audio-Visual  Specialist 

231 

116 

75130 

Apprentice  Education  Specialist 

10 

109 

75131 

Apprentice  Audio-Visual  Specialist 

88 

149 

75132 

Training  Specialist 

903 

686 

75390 

Small  Arms  Supervisor 

14 

13 

75370 

Small  Alma 

253 

579 

75330 

Small  Arms  Instructor 

780 

420 

Total 

4332 

4016 

Total  Minus  Small  Arms  AKST.a 

3285 

3004 

■Ah  ol  30  April  1966 

bl.niry  level;  Cnpuin  Soldi#*  *  position  ratlin*  lor  thr  rank  of  Major. 
'As  ol  3i  May  I5t6 


Service  Instruction  to  Prepare  Personnel  to  Develop  Curricula 

With  few  exceptions,  curricula  administered  to  first-enlistment  personnel 
are  prepared  by  instructors  selected  primarily  for  their  competence  in  the  sub¬ 
ject  matter  rather  than  in  training  concepts  and  techniques.  The  training  given 
military  curriculum  developers  would  benehi  from  updating  and  emphasis  in 
line  with  the  concepts  of  modern  training  technology;  such  instruction  is  espe¬ 
cially  important  in  view  of  the  fact  that  directives  and  guidance  that  would  imple¬ 
ment  the  steps  of  the  curriculum  development  model  are  not  provided. 
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Army.  Annex  Q  to  CON  Reg  350-1  (50)  requires  all  personnel  selected  for 
resident  instructor  duty  to  attend  an  instructor  training  course  prior  to  assign¬ 
ment  as  a  platform  or  shop  instructor.  Deviations  are  authorized  when  selected 
personnel  have  had  previous  training  and/or  experience  as  instructors.  Instruc¬ 
tor  training  is  not  standardized,  but  varies  with  the  individual  schools.  Courses 
usually  run  two  weeks,  with  prime  emphasis  placed  on  methods  of  instruction 
rather  than  content  determination.  The  basic  reference  is  usually  FM  21-6, 
Techniques  of  Military  Instruction  (89),  dated  May  1954.  Exceptions  to  this  pat¬ 
tern  include  the  instructor  training  course  at  the  U.S.  Army  Quartermaster 
School,  which  gives  heavy  emphasis  to  duty-oriented  objectives  for  guidance  in 
developing  course  objectives. 

Navy.  Naval  instructors  attend  instructor  training  courses  in  which  they  are 
required  (e.g.,  CNATECHTRA  Instruction  1540. 9G,  90)  to  (a)  acquire  a  working 
knowledge  of  the  fundamentals  of  teaching,  (b)  experience,  under  direct  super¬ 
vision,  the  preparation  and  presentation  of  lessons,  and  (c)  conduct  critical  anal¬ 
ysis  and  evaluation  of  lesson  presentations. 

Air  Force.  The  ATC  program  for  its  instructors  is  ATC  POI  AIR75100, 
Technical  Instructor  Course  (Technical  Training)  (91),  a  method s-of- instruction 
program  required  of  all  instructor  personnel.  In  addition.  POI  OZR7500-*, 
Development  and  Management  of  Training  Materials  (Technical  Training)  (92), 
is  available  for  mid-level  (officer)  supervisors  and  up.  The  course  runs  two 
weeks  and  includes  coverage  of  occupational  surveys,  preparation  of  JTSs,  POls, 
SOLOs,  and  Student  Study  Guides,  and  use  of  quality  control  techniques.  The 
accompanying  Student  Study  Guide  and  Workbook  includes  reprints  of  and  refer¬ 
ences  to  current  thinking  on  the  technology  of  training. 

Individual  ATC  centers  conduct  local  instruction  as  the  need  arises, 
including  workshops  on  the  preparation  of  Statements  of  Learning  Objectives. 

POI  AZR  75100,  Instructional  Programmer  (Technical  Training)  (93).  is 
used  by  the  Instructional  Systems  Branch,  Lackland  AFB,  to  instruct  in  methods 
of  programed  instruction. 

OiMeminotion  of  Data  far  Training  Purposes 

Among  the  agencies  that  provide  continuing  research  of  relevance  to  training 
for  the  services  are: 

Army 

The  Army  Personnel  Research  Office  (APRO),  Washington.  D.C. 

The  American  University  Center  for  Research  in  Social  Systems 
(CRESS,  formerly  SORO),  Washington,  D.C. 

The  Human  Engineering  Laboratories  (HEL),  Aberdeen  Proving 
Ground,  Maryland. 

The  George  Washington  University  Human  Resources  Research 
Office,  (HumRRO),  Alexandria,  Virginia. 

Navy 

The  Naval  Personnel  Research  Activities  at  San  Diego,  California, 
and  Washington,  D.C. 

Research  Facilities  at  Headquarters,  Naval  Air  Technical  Training 
Command,  Memphis,  Tennessee. 

The  New  Developments  Research  Branch,  Personnel  Research 
Division.  Bureau  of  Naval  Personnel,  Washington,  D.C. 

The  Naval  Training  Devices  Center,  Orlando,  Florida. 
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Air  Force 

The  Training  Research  Division.  Behavioral  Sciences  Laboratory. 
Aeromedical  Research  Laboratories.  Wright- Patterson  Air 
Force  Base.  Ohio. 

Personnel  Research  Laboratory,  Aerospace  Medical  Division. 
Lackland  Air  Force  Base,  Texas. 

There  is  a  continuing  need  in  all  services  for  greater  coordination  between 
the  efforts  of  personnel  research  and  training  application.  Often  data  that  would 
be  valuable  for  training  purposes  are  gathered  in  research,  but  provision  for 
transmittal  to  and  use  by  training  curriculum  development  personnel  is 
not  adequate. 

The  Air  Force  Personnel  Research  Laboratory,  for  example,  has  developed 
and  validated  px*>cedures  expressly  suited  tc  the  periodic  collection  of  military 
job  information.  That  laboratory  has  also  prepared  detailed  plans  for  the  estab¬ 
lishment  of  an  occupational  survey  unit  for  the  purpose  of  preparing,  adminis¬ 
tering,  and  analyzing  occupational  surveys,  but  no  requirement  was  found  for  the 
results  of  completed  surveys  to  be  utilized  by  Air  Force  training  personnel. 

A  source  of  job  information  available  to  Naval  training  personnel  is  the 
results  of  several  surveys  conducted  by  the  Naval  Personnel  Research  Activities 
(see  Navy  section  under  "Studies  Conducted  to  Obtain  Feedback,"  Chapter  4). 
Ratings  studied  have  included  Aviation  Structural  Mechanics.  Electronics  Tech¬ 
nicians,  Fire  Control  Technicians,  Shipfitters,  and  Sonarmen.  No  requirement 
was  found,  however,  for  the  findings  of  these  surveys  to  be  transmitted  to  and 
utilised  by  Naval  training  personnel. 


tote  of  Change  of  Service  Course* 

One  factor  in  considering  implementation  of  improved  technical  training 
curriculum  development  in  the  services  is  the  normal  rate  of  change  in  courses 
taught.  Data  collected  in  the  course  of  the  present  study,  although  incomplete, 
may  serve  to  outline  the  general  aspects  of  course  turnover. 

Data  collected  from  the  three  services  on  course  additions  and  deletions 
are  summarised  in  Table  5.  Overall,  it  shows  additions  at  about  13%  per  year 
and  deletions  at  about  10%  per  year.  This  rate  of  change  means  that  a  decision 
to  implement  the  model  training  development  process  only  in  new  course 
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development,  tor  example,  would  require  a  number  of  years  before  a  majority  of 
service  courses  were  job-oriented. 

Army.  Additions  and  deletions  in  Army  courses  in  DoD  Occupational 
Conversion  Table  categories  1,  2,  and  6  from  1961  through  i968  are  given  in 
Table  6.  There  were  additions  of  about  19%  for  the  five-year  period,  or  about 
4%  per  year.  Deletions  were  about  26%  for  the  five-year  period,  or  about  5% 
per  year. 

Navy.  Additions  and  deletions  in  courses  of  the  Bureau  of  Naval  Personnel 
and  Naval  Air  Technical  Training  Command  are  shown  in  Tables  7  and  6. 
Regarding  firm-enlistment  personnel,  the  data  on  Class  A  courses  is  of  greatest 
interest.  Normal  progression  through  Navy  courses  is  in  the  sequence:  Class  A, 
Cl***  C  Class  B.  (Data  were  not  readily  available  on  the  number  of  first- 
enlistment  personnel  in  Class  C  courses.)  The  tables  show  that  combined  addi¬ 
tions  of  Navy  Class  A  courses  were  about  11%  (or  the  five-year  period,  or  about 
2%  per  year.  Deletions  were,  for  the  five-year  period,  about  12%- again,  about 
2%  per  year. 

Air  Force.  Additions  and  deletions  of  Air  Force  ABR  courses  from  1961 
through  1965  are  shown  in  Table  9.  Further  information  wouid  be  necessary  to 
determine  what  proportion  of  Air  Force  courses  remain  unaffected  after  inter¬ 
vals  of  time,  but  the  data  suggest  rather  rapid  turnover.  Additions  for  the 
five-year  period  amounted  to  134%,  or  about  27%  per  year.  Deletions  for  the 
five-year  period  totaled  130%,  or  about  2®%  per  year. 

To  serve  as  guidance  for  decisions  on  how  best  to  implement  improved 
curriculum  development,  the  foregoing  information  on  course  turnover  in  the 
services  would  need  to  be  augmented  by  more  detailed  information.  The  criteria 
for  adding  and  deleting  courses  would  need  to  be  examined.  It  must  be  con¬ 
sidered.  too.  that  a  "new*  course  is  seldom  wholly  new  in  content,  and  hist 
course  revisions  may  be  minor  or  major  in  their  effect  on  course  content. 
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An  indication  of  the  kind  of  gains  that  have  been  realised  in  the  development 
of  job-oriented  curricula  is  given  in  Table  10.  The  table  shows  a  summary  of 
fin  J  mgs  on  course  length  and  proficiency  from  studies  that  HumRRO  has  made 
on  training  for  enlisted  Army  jobs  in  Cstegory  1  of  the  DoD  Occupational  Con¬ 
version  Table  (1).‘  The  findings  show  a  median  training  time  reduction  of  about 
six  weeks  in  a  group  of  studies  in  which  the  median  course  was  23  weeks  in 
length.  Coupled  with  this  result  for  the  cluster  of  studies  is  s  median  proficiency 
increase  of  23%,  The  overwhelming  majority  of  the  trainees  in  these  courses 
were  first -enlistment  personnel  low  exception  was  the  two-week  LORAS  course, 
in  which  the  trainees  were  experienced  petty  officers). 

The  Development  of  Instructional  Systems  (DIS)  approach  shows  sn  average 
course  length  reduction  of  one  to  two  weeks  (together  with  a  *9%  increase  in  job¬ 
relevant  content) and  a  33%  increase  in  proficiency.  Other  cost  benefits  indie  r  *d 
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Table  10 

HumRRO  Development  of  Job-Oriented  Training  Curricula 

Weeks  of  (‘our.et*  %  of 

Course  Content  — - —  -  — » —  Profirien<  y 


Conventional  j 

Kxprrimental  | 

lncre<t*>< 

Radio  Repair  (59) 

20 

20 

23 

Basic  Electronics  (97) 

3 

3 

5 

Radar  Repair  (_!) 

37 

22 

25 

Electronic  Maintenance  (26) 

30 

15 

10 

Electronic  Maintenance"  (98) 

2 

2 

200 

Radar  Repair  t5) 

30 

12 

0 

Basic  Electronics  (99) 

12 

6 

0 

Carrier  Equipment  Repair*1 

25 

11 

0 

Radar  Maintenance  (50) 

32 

26 

41 

8 A  reJuction  of  about  one  Hay  in  training  time  was  a*  hie\eH  *n  this  case. 

^Gebhard,  R.  “Development  and  Test  of  a  Training  Program  and  Job  Vds 
for  Maintenance  of  electronic  Communication  equipment,"  report  in  preparation- 

by  DIS  are  a  30-60%  reduction  in  OJT  time  and  two  to  three  weeks  (vs.  tradi¬ 
tional  12-15  weeks)  to  detect  course  failures. 

An  indication  of  other  benefits  from  job-oriented  training  is  found  in  an  Army 
magazine  article  by  Raymond  (96)  in  which  it  'S  stated  that  Radio  Mechanic 
Course  attrition  rates  were  cut  from  as  high  as  24%  to  less  than  2%  by  a  deci¬ 
sion  to  "cut  out  the  frills  and  the  non-essentials.  Teach  the  student  how  to  do 
his  job. "  DIS  personnel  report  unchanged  attrition  rates  despite  higher  standards. 

Costs  of  implementation  would  presumably  be  greatest  during  the  initial  and 
transitional  phases  of  re-orienting  military  training  curricula.  The  DIS  effort 
is  in  these  phases  at  present,  and  its  personnel  report  that  the  effort  has  been 
accomplished  within  r  al  resources.  DIS  comprises  only  a  very  small  seg¬ 
ment  of  military  instiuetion,  and  it  is  doubtful  wmether  its  cost  experience  can 
be  generalized  to  the  entire  military  establishment.  For  one  thing,  the  success 
of  DIS  has  been  highly  dependent  upon  command  support  at  the  Army  Security 
Agency;  similar  strong  approval  and  enthusiasm  could  not  be  expected  to  be 
generated  throughout  the  services,  and  the  cost  of  such  eiforts  is  doubtless 
related  to  the  drive  behind  and  within  them.  Nevertheless,  the  DIS  experience 
suggests  that  costs  might  well  be  within  manageable  limits.  Once  the  new  sys¬ 
tem  is  fully  operational,  the  weight  of  research  evidence  suggests  that  costs  of 
operation  should  be  less  than  before. 


Chapter  6 


PRINCIPAL  FINDINGS 


The  report  presents  a  model  training  curriculum  development  process  con¬ 
sisting  of  the  following  steps  (fully  described  in  Chapter  2):  (a)  conduct  system 
analysis,  (b)  develop  task  inventories,  (e)  develop  job  model,  (d)  conduct  task 
analysis,  (e)  derive  training  objectives,  (f)  develop  training  program,  (g)  moni¬ 
tor  trained  product  and  modify  training  curriculum  as  required.  A  number  of 
the  findings  and  conclusions  relate  to  the  model. 

The  principal  findings  may  be  summarized  as  follows: 

(1)  Training  Objectives.  The  first  four  steps  of  the  model  development 
process  were  performed  in  a  minimal  way  or  not  at  all  by  the  military  services. 
Consequently,  although  all  services  referred  to  the  need  for  job  relevance, 
training  objectives  were  not  satisfactorily  tied  to  system  and  job  requirements. 
The  services  would  benefit  from  procedural  guidance  and  directives  for  devel¬ 
oping  or  conducting  the  first  four  steps. 

(2)  Allocation  of  Content  to  Formal  Training  or  On-Job  Learning. 
Except  at  two  Army  schools,  guidance  on  allocation  of  content  was  not  adequate, 
an.  .ere  was  no  requirement  for  statement  of  criteria  for  allocating  subject 
matter  to  formal  training  or  to  on- job  training.  A  statement  of  criteria  is  needed. 

(3)  Specification  of  Graduate  Capability.  The  first  requirement  for 
quality  control  is  product  specification.  The  nearest  approach  to  an  adeqi  ate 
certification  of  graduate  capability  was  the  Air  Force  Job  Training  Standard, 
but  it  lacked  specificity  and  an  adequate  analytic  basis.  Precise  graduate  spec¬ 
ifications  are  needed. 

(4)  Feedback  from  Job  to  Training.  The  services  generally  endeavor 
to  obtain  formal  or  informal  job  performance  information  on  graduates.  The 
principal  means  used  was  the  mailed  questionnaire;  this  method  is  inexpensive 
but  provides  data  of  inferior  quality.  The  Air  Forcp  alone  scheduled  routine 
field  visits  to  obtain  feedback  information.  All  the  services  would  benefit,  from 
routine  field  visits. 

(5)  Advancement  Test  Data.  The  data  generated  by  existing  service 
evaluation  and  advancement  testing  procedures  appear  to  be  of  doubtful  value 
for  gauging  training  effectiveness. 

(G)  Application  of  Model  Curriculum  Development  Process.  A  program 
at  the  USASATC&S  is  the  only  instance  found  where  a  complete  application  of 
the  model  curriculum  development  piocess  was  in  progress.  USASATC&S  per¬ 
sonnel  reported  that,  in  initial  course  work,  average  course  length  decreased 
and  content  increased,  graduates  were  better  qualified,  attrition  rates  were  the 
same  despite  higher  standards,  and  trainee  motivatio  improved.  Consonant 
with  these  findings,  HumRRO  job-oriented  training  developments  in  studies 
dealing  with  electronic  equipment  repairmen  jobs  have  resulted  in  a  reduction 
of  about  one  fourth  (median)  in  course  length  and  in  an  increase  of  about  one- 
fourth  (median)  in  proficiency. 
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(7)  Service  Career  Fields  for  Training.  Although  training  costs  account 
for  6%  of  the  Defense  Budget,  only  the  Air  Force  had  a  career  field  for  training, 
and  the  personnel  in  that  field  represented  less  than  1%  of  Air  Force  strength. 

All  the  services  need  a  training  career  field  more  nearly  in  proportion  to  the 
importance  of  training  as  a  military  activity. 

(8)  Training  for  Curriculum  Development.  The  typical  curriculum 
developer  is  an  instructor.  All  services  provided  instructor  training,  but  the 
primary  emphasis  in  that  training  was  on  methods  of  instruction,  not  on  content 
determination.  The  Air  Force  alone  among  the  services  provided  a  course  ir 
curriculum  development,  and  it  was  a  two-week  officer  supplemental  course. 
Adequate  training  in  the  steps  of  the  curriculum  development  process  is  essential. 

(9)  Coordination  Between  Research  and  Training  Development  Agencies. 
Better  provision  needs  to  be  made  for  training  research  information  to  be  trans¬ 
mitted  to  and  used  by  training  curriculum  development  personnel. 

(10)  Rate  of  Change  of  Service  Curricula.  The  rate  of  change  (addition 
and  deletion)  of  service  courses  requires  further  study,  but  the  available  infor¬ 
mation  suggests  that  adoption  of  job-oriented  analysis  procedures  solely  for  new 
courses  would  mean  that  a  number  of  years  would  pass  before  a  majority  of 
service  courses  were  job-oriented. 
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GLOSSARY  Of  TIRMS  AND  AtMCVIATIONS 


ABB  Cowm 
ATS 

AFSC 

ATC 

Btpm 
COTMT  Field 


CER 

CMATECHTRA 

CTEP 

OB 


EDR 

EEC 


t«k  Aneiyni* 
Jot  Model 

m 

ItoefcM  Nik 

BOS 


Atman  Baste  Resident  Courr*.  U  S.  Air  Force. 

Air  Force  Specially.  An  occupational  grouping  suular  to  the  Army  UCS  md 
the  Navy  rating. 


Aw  Force  Specialty  Cod?  A  *nd*gtt  code  designating  an  Aw  Farce 
duty  politic*.. 

Aw  Tratmft;  Coen  and. 

Bureau  o!  Nang!  P<Mior.ae!. 

A  hwkx  occupational  grouping  within  4  military  .at  structure  The  Air  Force, 
for  example,  ha*  IS  Officer  Career  Area;  rra  J6  £nhr*~t  Career  FieH*. 

Comeandet  Evaluation  Report.  Sating*  an  Army  enlisted  personnel. 

Chief  of  Naval  Ar  Techr.ts.ai  T: aining 
CorvaoUdared  Training  ana  Education  r.fajr«n. 

Tevelcpoent  a£  Instructional  System*.  A  trauuna  curnculu.-?  development 
pracfi*  in  use  4  ’.he  U4-  Armv  Security  Agency  Ttamin*  Cent*?  *4  School, 
i  oft  P*«*tn*.  Ma**<JL*hu-et*  •-.. 

F-vaiu'Otim  Cat  a  Repotf.  Report  -"jr  Army  enlisted  p^tnonne*  ey  A*siy  Enlisted 
EV liajrtan  Center. 

Enlisted  Evaluaunn  An  Army  t*ili»y.tlt*  njsunwitet^  te.-»*  and 

»  Uect ;  dm  3a  cfilist.rej  pctisjofsei  ret  ftsticteficy  evaluation 

Us  tstisinj  deveiapmest  trie  pfooes?  e*  j4hennj  miotmatijn  an  ;*-*$.,*<-  >st 
ferfetmaBk?#-  f;*'  pcsxifcis  use  in  revision. 

A-  in  this  report.  f>-oce  juf-5  far  gothetsrt*  istoiteaf'Oft  leading  to  the  eon 
^ifyCHa**  of  1  1*3 1>  c.e*dri 

A  Srft  01  di-‘*M  **k  drifting  the  pat-  petfgrm-Jnc-*.*  toward  *fe<h 

ig  t§  jfc«  degigaed 

Jdfc  Training  St-.*=d*d  jAir  Faery>. 

A  S*  ptentt 2t  o* dcf WM  ef  *e  Jfkfccp  gf  tie  :pfe**rt  of  ■$  Solitary  Surfs** 
i«  J  .  in  the  CSs*  Of  ‘*r{?4;ye  §t*'rgtioS  *'•  ttigiUgft.  ifei  ??<*£*.  i?=<dl>3t! 

tdr  :.e!  tire!.  *;*hsn  *=*Hn  uggise  pn>di».  4  m&mrtm  SI  *4*ir 

ppff  *i  the  tot  tied  »i?h  pgrtscusir  to  ?i#rh 

f*.  *3r  j  SfitjeOf  fsswi  ^Ort^-gr^rsni  ji  giflueft:.  *i  *!ii  fe  Jetiseghjp  to  the 

■  IjV?  of  e*h»rr  pet*  in  tftr  tsinsvgn 

Vf: I;* sr,  Ceo-ip^tionai  Sprcigjty  'A-sr'  As  ovrusatfsnsl  sr-rtofe-i  psssuir  *e 
the  Vjrr  eftSift-j  gad  '  he  AF  T  -of  "tue  Ar  5* -e*re 


NATECKTBACGM  Nava*  An  TecUhicii  Truistisa  ‘  -«s.-  •*.$ 

RAVPERS  Bure-Jt*  3f  Jis.  ii  f-orj.rnsei 


NCO 

NEC 

NOBC 

OJT 

POI 

QQPP.I 

"Quals" 

Rating 


SKT 

SOLO 

System  analysis 


Task  analysis 


Task  description 

Task  inventory 
TED 


Noncommissioned  officer. 

Naval  Enlisted  Classification. 

Nnval  Officer  Billet  Code.  A  Naval  sub- specialization,  not  career -oriented. 
On-job  training. 

Program  ot  Instruction  (Army);  Plan  of  Instruction  (Air  Force). 

Qualitative  and  Quantitative  Personnel  Requirements  Information. 

Common  expression  for  Qualifications  for  Advancement  in  Rating  (Navy). 

A  Naval  enlisted  occupational  qrouping  similar  to  the  Army  MOS  and  the  Air 
Force  AFS- 

Specialty  Knt.viedqe  Test.  An  Air  Force  test  for  advancement  within  an  AFS. 

Statement  ci  L'-a*n;ng  Objective.  An  Air  Force  training  objective. 

A  study  of  the  operational  s-  stem  ol  which  the  job  is  a  part,  its  missions, 
functions,  and  environments,  in  order  to  establish  the  relationship  of  the  job 
to  the  system  for  traininq  development  purposes. 

A  set  of  procedure  for  detailed  ~tudy  o!  tasks,  used  for  the  purpose  oi 
providing  a  job-valid  basis  for  (a)  allocating  tasks  to  formal  traininq  or  to 
on-job  learning  ar.d  lb)  the  derive* ior.  ol  traininq  objectives. 

A  statement  of  the  performances  involved  in  the  accomplishment  of  a  task, 
sufficiently  detailed  for  the  purpose  oi  task  analysis. 

An  organized  list  of  duties  and  tasks  performed  by  personnel  on  a  job. 
Traininq  Evaluation  Division.  An  Air  Force  facility  responsible  for  feedback 
efforts  in  a  traininq  center. 


TOE  Table  of  Organization  and  Equipment.  Term  often  used  to  refer  to  an  Army  unit 

with  a  general  mission,  in  contrast  to  TD  (Table  of  Distribution),  used  to  refer 
to  units  with  special  missions. 

Training  Objective  A  precise  statement  of  a  discrete  performance  to  be  learned  in  formal  training. 

The  statement  should  include  definitions  of  what  performance  is  tc  be 
lea.ned,  the  conditions  under  which  it  is  to  be  demonstrated,  and  the 
standards  fc  acceptable  performance. 


USAADS 

USAAMS 

USAARMS 

USAAVNS 

USACSS 

USAES 

USAINTS 

USAIS 

USAMMCS 

USAOC&S 

USASESCS 

USASCS 


U.S.  Army 
U.S.  Army 
U.S.  Army 
U.S.  Army 
U.S.  Army 
U.S.  Army 
U.S.  Army 
U.S.  Army 
U.S.  Army 
U.S.  Army 
U.S.  Army 
U.S.  Army 


Air  Defense  School,  Fort  Bliss,  Texas. 

Artillery  and  Missile  School,  Fort  Sill,  Oklahoma. 

Armor  School,  Fort  Knox,  Kentucky. 

Aviation  School,  Fort  Rucker,  Alabama. 

Combat  Surveillance  School,  Fort  Huachuca,  Arizona. 

Engineer  School,  Fort  Belvoir,  Virginic. 

Intelligence  School,  Fort  Holabird,  Maryland. 

Infantry  School,  Fort  Banning,  Georgia. 

Missile  and  Munitions  Center  and  School,  Redstone  Arsenal,  Alabama. 
Ordnance  Center  and  School,  Aberdeen  Proving  Ground,  Maryland. 
Southeastern  Sianal  School,  Fort  Gordon,  Georgia. 

Signal  Center  and  School,  Fort  Monmouth,  New  Jersey. 
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USASATC&S 

USATSCH 

USAQMS 

USCOfcARC 


U.S.  Army  Security  Agency  Training  Center  and  School,  Fort  Devons, 
Massachusetts.. 

U.S.  Army  Transportation  School,  r'ort  Ejstis,  Virainia.  . 

U.S.  Army  Quartecmastei  School,  Fort  Lee,  Virginia. 

U.S.  Continental  Army. Command,  Fort  Moriroe,  Virginia. 
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Appendix  A 


DEVELOPMENT  OF  USCONARC  REGULATION  ON 
SYSTEMS  ENGINEERING  OF  TRAINING 


DEPARTMENT  OF  THE  ARMY 

HEADQUARTERS  UNITED  STATES  CONTINENTAL  ARMY  COMMAND 
FORT  MONROE,  VIRGINIA  23351 


4  June  1968 

SUBJECT:  Draft  HumRRO  Technical  Report 

Chief  of  Research  and  Development 
Department  of  the  Army 
ATTN:  CRDBES 
Washington,  D.C.  20310 


1.  Reference  is  made  to  draft  HumRRO  Technical  Report,  The  Curriculum  Content 
Study  Area  of  the  Consolidated  Training  and  Education  Program  (CTEP)  at 
inclosure  1. 

1.  This  headquarters  recently  published  a  regulation  CON  Reg  350-100-1,  "Systems 
Engineering  of  Training  (Course  Design),1'  which  is  related  to  the  CTEP  Study.  A 
copy  of  the  regulation  is  at  inclosure  2. 

3.  A  team  composed  of  representatives  from  six  CONARC  service  schools,  a  repre¬ 
sentative  from  USCONARC  and  a  consultant  (Dr.  Taylor)  from  HumRRO  worked  for 
almost  one  year  to  develop  this  regulation.  During  this  time,  all  known  publi¬ 
cations  dealing  with  systematic  course  design  were  reviewed  and  those  articles 
considered  appropriate  to  the  USCONARC  training  situation  were  used.  Some  of 
the  more  profitable  publications  were  the  US  Army  Security  Agency's  "Project 
MINERVA,"  and  HumRRO  articles  by  Ammerman  and  Smith.  Principle  use  was  made  of 
the  publication  by  Smith,  "The  Design  of  Instructional  Systems,"  Human  Resouces 
Research  Office,  1967.  Incorporated  into  the  USCONARC  regulation  are  the  seven 
steps  of  the  model  process  for  training  curriculum  development  recommended  in 
the  CTEP  study,  although  entitled  differently.  The  regulation  also  requires  all 
USCONARC  schools  and  training  centers  to  systems  engineer  all  MOS  producing 
courses,  functional  courses,  career  courses,  and  Army  Subject  Schedules,  for 
which  proponent.  It  is  expected  that  such  an  operation  will  be  accomplished 
over  a  five  year  period  beginning  1  April  1968. 

4.  The  CTEP  study  recommendation  that  the  services  provide  directives  and 
detailed  procedural  guidance  for  developing  and  conducting  systematic  course 
design,  or  redesign,  has  been  recognised  by  USCONARC  with  the  publication  cf 
CON  Reg  330-100-1. 

FOR  THE  COtffANOER: 


2  tncl 

as 


it 


Copy  furnished: 
Oir,  HumRRO 


D.A.  FOUERSON 
Major,  ACC 
ASSt  AC 


Appendix  B 


SUMMARY  OF  US  ONA»C  SCHOOLS’  FEEDBACK 
TECHNIQUES  FOR  ENi.!Sf£D  TECHNICAL  COURSES 


Guidance  to  Army  schools  on  feedback  it,  contained  in  Annex  Q,  Army 
Service  Schools  Curriculum  Administration  and  Training  Policies,  to  CON 
Reg  350-1,  USCONARC  Training  Directive,  18  May  *  S*'5  :r0).  Under  II  Students, 
5.  Service  school  responsibilities,  b.  Reduction  oi  attr  ao>i.  Annex  Q  states, 
"Commandants  of  schools  will: 

(d)  use  feedback  data  from  course  graduates  and  their  s:,pt  -visors  to 
evaluate  courses. 

1_.  Schools  may  send  questionnaires  to  graduates  and  their  imme¬ 
diate  supervisors  after  the  school-trained  skills  have  been  used  on  the  job  for 
approximately  3  to  6  months  to  obtain  feedback  data.  The  use  of  questionnaires 
is  most  beneficial  with  large  input  courses  where  a  high  return  of  usable  ques¬ 
tionnaires  can  be  expected.  When  questionnaires  are  used,  direct  mailing  of 
the  questionnaires  and  follow-up  letters  containing  additional  copies  of  the  ques¬ 
tionnaires  to  those  responding  provides  the  greatest  return.  Questionnaires 
will  include,  but  will  not  be  limited  to,  items  designed  to  determine  the  diffi¬ 
culty  and  the  frequency  of  specific  job  tasks  performed  by  the  graduates. 

2.  The  onsite  observation  and  interrogation  of  school  graduates 
and  their  supervisors  in  their  work  assignments  is  the  most  valid  technique  for 
obtaining  feedback  data." 

In  following  this  guidance,  the  14  Army  schools  engaged  in  enlisted  technical 
training  for  DoD  Occupational  Conversion  Table  categories  1,  2,  and  6  employ 
the  techniques  described  below. 

USAADS  (Air  Defense  School) 

Conducts  interviews  with  personnel  from  units  returning  to  Fort  Bliss  for 
Annual  Service  Practice.  Interviews  nre  conducted  to  determine  the  adequacy 
of  school  graduates,  to  solicit  constructive  criticism  from  school  graduates, 
and  to  determine  the  adequacy  of  technical  manuals  and  training  literature  for 
which  the  School  is  the  proponent  agency.  A  questionnaire  program  is  being 
developed 

USAAMS  (Artillery  and  Missile  School) 

Administers  cnd-of-course  questionnaires  to  resident  classes  and  follow¬ 
up  questionnaires  to  graduates  and  immediate  supervisors  after  the  graduate 
has  been  on  the  job  four  to  six  months- 


u 


Example  of  follow-up  questionnaire  to  graduate: 


(1) 

KNOWLEDGE  AND 
SKILL  AXEA 


JOB  SKILLS 


BASIC  ELECTRONICS 


Mathematics 


Electricity 

Electronics 


Elect.  Warfare 


Radar  Funda¬ 
mentals 


COMPARED  WITH  THE  AMOUNT  NEEDED  TO  ADEQUATELY 


PREPARE  FOR  JOB 


(2) 

What  degree  of 
EMPHASIS  (time, 
effort,  etc.)  was 
provided  by  the 
course  for  each 
area  in  Column  1? 


Too 

Much 


About 

Right 


Too 

Little 


(3) 

What  amount  of 
PRACTICAL  WORK 
(lab,  field  exer- 
cises,  etc.)  was 
provided  in  the 
course  for  each 
area  in  Column  1? 


Too 

Much 


About 

Right 


Too 

Little 


(«) 

Mat  amount  of 
CLASSROOM  INSTRUC¬ 
TION  (theory,  sub¬ 
ject  matter,  etc.) 
was  provided  In  the 
course  for  each 
area  In  Column  1? 


Too 

Much 


About 

Right 


Too 

Little 


Example  of  immediate  supervisor  questionnaire: 


L--'  ARMS  (Armor  School) 

Send*  questionnaire!!  to  commanders  and  immediate  supervisors  of  graduates 
Example  of  questionnaire  to  commander: 

3.  Upon  Interviewing  the  soldier,  did  he  display  self -confidence  In  his  ability 
to  perfora  In  his  NQ$? _ _ 


4.  Evaluate  the  soldier  as  to  his  proficiency  or  shortcomings  In  the  areas  out¬ 
lined  below. 

a.  Type  of  vehlcle(s)  he  Is  maintaining. _ _ _ _ 


b.  Application  of  the  A rqy  Equipment  Records  System  and  Procedures. 


c.  Use  of  technical  manuals,  lubrication  oroers.  and  otner  publications  and 
directives  pertinent  to  organization  maintenance. _ 


Example  of  questionnaire  to  immediate  supervisor: 


Job  Skills 

Oagraa  of  Proficiency  Displayed 

!  Ro  Oppor¬ 
tunity  to 
Observe 

High 

Moderate 

Satisfactory 

r- 

1.  Road  2  understand  sche¬ 
matic  f  block  diagrams. 

i 

2.  Use  technical  manuals  t 
maintenance  '„vb  11  cat  ions. 

i 

L _ 

^  J  1;"  1  rTliMHi 

r 

4.  Apply  troubleshooting 
wocedures-AR  Radio  lets. 

$.  <ipp!y  troubleshooting 
trocadures-nt  Radio  Sets. 

. 

r 

s 

*-  ihowsignal  supply  pro- 

1 

■ 

| 

- . i 

i 

i 

| 

7.  ■erforn  organizational 
mintenance. 

j 

r  ! 

l 

i 

1 

i- 

i 

| 

4.  now  welt  trained  was  this  a*a  as  a  result  of  Ms  attending  the  Ceamwat cation 
Specialist  Course? 

_ very  well  trained 


Poorly  trained 


USAAVNS  (Aviation  School) 


Sends  questionnaires  to  graduates  and  supervisors- 
Examples  from  graduate  questionnaire* 


0  -  I  have  never  perfrnaed  this  ask. 

1  -  I  have  performed  this  tes*  tvj. 

?  -  I  have  perfor  m  this  tc?k  »  to  4  tines. 

3  -  l  have  perfomed  this  task  nere  than  4  tines. 

4  -  I  have  assisted  in  performing  this  task  once. 

5  -  I  have  assisted  in  perfoming  this  task  sore 

than  ^nce. 

TASK  US?  FOR  671.*  MECHANIC 

i 

2.  Daily  Inspection,  all  system 

3.  Interned late  Inspection,  all  syst ew 

4.  Periodic  Inspection,  all  sys^j 

5.  Operational  Check  for  a  Specific  Deficiency 


0  -  Remain  the  sane 
1  *  Increase  1  hour 
?  -  Increase  2  hours 

3  *  Increase  3  hours 

4  *  Increase  s  hours 


$  *  Decrease  1  hour 

6  *  Decrease  2  hours 

7  •  Decrease  3  hours 
*  -  Decrease  4  hoerj 


Example:  If  you  wish  to  Increase  number  41  by  4  hours,  nark  the  4  po,i -  on 

SirSL4  .^  Cir4'  Th*n  r#rfuc#  *uhjacts  by  4  hoursV  indl 
eating  then  In  the  sane  manner. 


SUBJECT  -  0-1 

40.  Description 

41.  Aircraft  Toois 

42.  Aircraft  Technical  Publications.  Form,  and  Records 

43.  Ground  Handling  Servicing  end  Engine  R*n>up 

44.  Unding  Geer  and  Brake  Systtn 
4$.  Fuel  and  Induction  System 
44.  Ignition  System 


HOURS 


Examples  from  supervisor  questionnaire: 


1. 


1  -  Not  applicable  for  this  graduate. 

2  -  A  weak  point  for  this  graduate. 

3  -  Graduate's  proficiency  is  average  In  this  area. 

4  -  A  strong  point  for  this  graduate. 

5  -  I  have  not  observed  the  graduate  in  this  area. 

2.  Dally  Inspection,  all  system 

3.  Intermediate  Inspection,  all  systems 

4.  Periodic  Inspection,  all  systems 

5.  Operational  Check,  any  system 

6.  Lubrication  In  accordance  with  daily  inspection 

7.  Lubrication  In  accordance  with  Intermediate  Inspection 

8.  Lubrication  in  accordance  with  periodic  Inspection 


2. 


0  •  Remain  the  same 

1  -  Increase  1  hour 

2  -  Increase  2  hours 

3  -  Increase  3  hours 

4  -  Increase  4  hours 


5  -  Decrease  1  hour 

6  -  Decrease  2  hours 

7  -  Decrease  3  hours 

8  -  Decrease  4  hours 


Example:  If  you  wish  to  increase  number  60  by  4  hours,  mark  the  4  position 
for  mmber  60  on  the  answer  card.  Then  reduce  other  subjects  by  4  hours.  Indi¬ 
cating  them  In  the  seme  manner. 


SUBJECTS  HOURS 

56.  OH-13  Structure  1 

57.  Helicopter  Aerodynamics  1 

58.  OH- 13  Fuel  and  Oil  Systems  1 

59.  OH-13  Power  Transmission  System  1 


USACSS  (Combat  Surveillance  School) 

tends  -#uttea*e“  team*  to  organisation*  rrqur#ttng  additional 
information  and  guidance  on  combat  *urve»tlance  equipment  The  primary  pur* 
po*e  of  three  team*  i»  to  di##emi«ate  information  and  correct  deflriencie*  a* 
they  are  found  in  the  field  \n  evaluation  of  equipment  and  peraonnel  **  utually 
made  before  mttitvr  can  be  effectively  rrndt  rd  Although  field  evaluation# 
are  not  formally  conducted,  the  information  determined  from  the  suite  axe  team# 
t#  used  in  the  game  manner  a*  would  be  the  information  from  a  formal  field 
evaluat.on  In  *«mc  area#,  the#e  suite  »#e*  team  evaluation*  are  felt  to  be  more 
accurate  beeauac  they  mutt  become  mo-  of  *h*»  a  unit  ha#  done  be*ofe  they 
can  begin  to  attist  n 


USACSS  S‘?nds  questionnaires  to  graduates  and  supervisors. 
Example  oi  q.u  stionnaire  to  graduate: 


stems 

FREQUENTLY 

OCCASIONALLY 

NOT  AT  AIL 

Operated  and  performed 
maintenance 

Operated  and  performed  maintenance 
using  Indicator  Test  Set  AN/6PM-41 
and  AN/GPH-52 

Aligned  the  system 

Operated  and  performed  maintenance 
on  the  recorder 

Example  of  questionnaire  to  immediate  supervisor: 


ob  Skills 

Degree  of 

Proficiency  Displayed 

No 

Opportunity 
ta  Observe 

High 

Moderate 

Satis¬ 

factory 

Low 

A.  Operational  Skill 

B.  Analyzing  Symptoms 

C.  Proper  use  of  Test 
Equipment 

U.  Isolating  Malfunctions 
Down  to  an  individual 
Component 

E.  Speed  of  Correcting 
Malfunctions 

f 

fc  -  i 

r-.  < 

r  ■ . 

t 

l 

f 

r  *  • 

f 

r-  -  ■  •  •  j 

I  | 

i 

1 

■  . . 

USAES  (Engineer  School) 

Evaluations  of  job  requirements  and  job  performance  of  recent  graduates 
mode  by  staff  and  faculty  replacements  recently  returned  from  field  units  and  by 
officer  students  enrolled  in  career  courses  serve  as  additional  sources  of  data 

USAES  sends  questionnaires  to  graduates  and  supervisors  after  the  gradu¬ 
ate  has  been  on  the  job  3  to  6  months. 

Example  of  questionnaire  to  graduate: 


1 

1.  Have  you  classified  or  Identified 
minerals  and  rocks? 

COLUMN  1 

COLUMN  II 

How  often  do  you 
perform  this  task? 

1 

(Check  One) 

Was  your  school 
training  adequate 
for  this  task? 
(Check  One) 

Fre¬ 

quent* 

ly 

Occa¬ 

sion¬ 

ally 

Never 

YES 

NO 

1 

i 

i  .  J 

i 

I 

• 

2.  Have  you  performed  or  used  the 
following  soils  tests? 

a.  Sieve  Analysis 

| 

i 

| 

| 

b.  Specific  Gravity 

f-- 

1 

r  "  ■■ 
\  -  -  -- 

1 _ 

c.  Moisture  Content 

d.  Wet  Mechanical  Analysis 
(Decantation) 

r  f 

1 _ 

' 

3.  The  questionnaire  is  designed  for  use  with  FH  Form  1026  (IBM  Sheet),  a 
copy  of  which  is  attached.  Section  II  of  the  questionnaire  is  a  summation  of 
subjects  taught.  Place  your  answers  on  FH  Form  1025.  using  the  following  key  for 
recording  your  answers:  (Multiple  answers  are  acceptable;  however,  please  do  not 
use  items  d  and  e  unless  you  have  previously  used  either  a.  b,  or  c  In  the  same 
question. ) 

a.  Must  know. 

*j.  Nice  to  know. 

c.  No  need  to  know. 

d.  Increase  Instruct*  ial  hours. 

e.  Decrease  Instr.ttt:  il  hours. 

SECTION  II  -  ACADEMIC  EVALUATION  .Answer  on  FH  Form  1026) 

A.  ORIENTATION  SUBJECTS.  VIETNAM. 

!  ACS1  Briefing  (Guest  lecture). 

2.  Pre-Departure  Personal,  Legal  Affairs  (Guest  lecture). 

3.  Republic  of  Vietnam  Intelligence  and  Security  Agencies. 

US  Organization  in  Republic  of  Vietnam. 

5.  Advisor  Communication  Problems. 


'  1$:' * 5- Ufita,drv  School) 

(No  examples  of  questionnaires  for  enlisted  graduates  were  received 

from  I  TSAIS. ) 

USASCS  (Signal  Sohoo!) 

Newly  assigned  personnel  are  inter/tewed  and  administered  questionnaires. 
USASCS  sends  questionnaires  to  graduates  and  supervisors 

Example  of  questionnaire  to  graduate: 


at 


Example  of  questionnaire  to  supervisor: 


SCALE  VALUE 


Not  Ob¬ 
served  N.O. 


U  >» 

<*-  v 
o 

u 

— - 

>  o 
«  w 


COL. 


STANOAROS  AND  DEFINITIONS 


Have  not  observed  repairman  perform  duties  in  this  major  area 


Competence  is  Limited:  Requires  detailed  guidance  and  close 
supervision,  Steeds  extended  on-the-job  training. 


Is  Moderately  Competent:  Requires  some  guidance  and 
supervision,  mainly  on  new  equipment  and  more 
; difficult  tasks. 


Is  Competent:  Can  perform  "on  his  own"  unless 
special problems  are  encountered.  Only  a  general 
check  of  his  work  by  the  supervisor  is  required. 


Is  Hit 

and  efficiently,  and  can  apply  correct 
procedures  and  techniques  to  new  tasks  or 
equipment. 


Highly  Competent:  Performs  skillfully 
efficiently, 


MAJOR  AREAS 


N.O. 


18 

19 


1.  SUPPLY  (Identifying  and  requisitioning 
parts,  etc.)  . 


2.  ELECTRICAL  FUNDAMENTALS  (Applying  laws, 
bas*c  measurements .  etc.)  . 


A 

A 


C 

<D 


0 

0 


E 


USASESCS  (Southeastern  Signal  School) 

Sends  questionnaires  to  graduates  ana  supervisors  (same  formats  as 
LTSASCS)  Also  administers  questionnaires  tj  incoming  enlisted  and  officer 
personnel,  and  queries  field  commanders  by  commend  letter 


USAOCtfS  (Ordnance  School) 

Makes  field  visits.  Sends  questionnaires  to  graduates  and  supervisors. 


Fxample  of  field  interview  form: 


CARD  2 

r— - 

f 

— 

0/1 

TBA 

sta| 

j 

■ 

m 

HI 

r  . .  “  ITEM 

C 

5 

a 

B 

a 

A.  AUTOMOTIVE  1.  Supervise  Safety  Prog. 

9-11 

MAINTENANCE  2.  Identify  Mil.  Vehicles 

12-14 

MANAGEMENT  3.  Asgn.  Duties  to  Prsnl. 

T5-T7 

- 

4.  Org.  A  Asgn.  Shop  Tunc. 

18-25 

5.  Org.  Maint.  Section 

21-23 

6.  Orq.  fixed  Shop 

24-26 

T.  Investigate  Deadline 

17-W 

8.  Supervise  Sup.  Func. 

9.  Interpret  MwO,  TB,  UER's 

10.  Org.  Tech.  Asst.  Teams 

3fF32~ 

33-35 
36  38 
19^4T 
42-44 
45-47 

' 

11.  Supervise  Storage 

’  12'  Supervise  Pail  Loading 

13.  Prepare  SOP's 

— 

— 

— 

14.  Perform  Liaison 

48-60 

r  15.  Supervise  Sched.  Maint. 

16.  Supervise  Spot  Check  Insp. 

17.  Supervise  Ovr.  Selec.  A  Tng. 

18.  Supervise  Care  A  Use  of  tools 

19.  Supr.  Use  A  Care  of  Pub.  A  FMs." 

[’  B.  TECHNICAL  1.  Supervise  Engine  Repairs 

51-53 

64-56 

57-59. 

60-62 

63-65' 

66-68 

— 

— 

— 

_ 

— 

— 

SUPERVI-  2.  Supervise  Pow^r  Train  Repairs 

69-7T 

!  SION  3.  Supervise  ruel  A  Elect.  Reprs. 

r  4.  Supervise  Recovery  Opns. 

\  ... 

C  ■  Iffi  Column  Number 

F  »  Number  of  Times  in  Typical  30-Day  Period.  A  ■  Nev 
0/1*1  Observed.  2  Interviewed. 

TBA  •  Training  Best  Accomplished.  4  School.  5  OJT. 
i  STA  *  School  Training  Adequate.  7  Yes.  8  No.  9  Excess 

72-7T' 

75-77 

er.  J 

ive. 

>  No 

t  Suj 

ppor 

ted. 

I  'cample  of  questionnaire  to  graduate: 


u 


uOi,  TASKS 


Rifles 

Pistols 


PERFORM  VISUAL  INSPECTION 

SECTION  1  [  SECTION  2 

In  the  post  4  months  I  When  I  Old  this  U$k 
I  have  done  this  usk  I  I  found  it  to  bt 

f  ?  :  I 


i/1 


I  si  si  ;$ 

i  X  1  **  :  f*l'  •*<  ••  « 


|  |£*sy  |  Difficult |  N/A 

5  i  i 


z  i 

©  | 

li 


i 

ft- 


-f 

± 


~s 


4 


UL 


u 


Example  of  questionnaire  to  supervisor: 


1.  Rifles 

PERFORMING  V 

SUAL  INSPECTION 

SECTION  1 

SECTION  2 

Is  this  task  essential  for 
carrying  out  the  mission 
of  this  unit? 

When  doing  this  task  most  of 
the  time  he: 

Ves 

No 

Not  Sure 

Needs  help 
from  others 

Acts  Inde¬ 
pendently 

N/A 

i/ 

\S 

USAMMCS  (Missile  and  Munitions  School) 
j  Makes  field  visits  Sends  questionnaires  to  graduates  and  supervisors 

Example  of  questionnaire  to  graduate: 


TEST  STATION  REPAIRMAN  (SERGEANT)  -  MOS  375 

SUBJECTS 

COMMENTS 

1. 

Basic  shop  practices 

1. 

No  change  recommended 

2. 

Use  of  TM's  and  supply  manuals 

2. 

Not  required  in  the  field 

3. 

Basic  supply  manuals 

3. 

Not  taught  but  needed 

4. 

Electrical  fundamentals 

4. 

More  classroom  Instruction 

5. 

Electronic  fundamentals 

5. 

More  practical  Instruction 

6. 

Common  guidance  and  control 

6. 

More  troubleshooting  practice 

7. 

OMTS  (overall) 

7. 

More  circuit  analysis 

8. 

Programming  system 

8, 

School  instruction  not  In  agree- 

9. 

Monitoring  system 

9. 

ment  with  field  application 

10. 

Tolerance  verification  system 

School  troubleshooting  did  not 
help  me  in  the  field 

11. 

Testing  system 

10. 

Practical  exercises  did  not 

12. 

FMTS  (overall) 

prepare  me  for  the  work 

». 

Test  control  and  tape  system 

encountered 

14. 

Power  and  measuring  system 

11. 

Less  classroom  Instruction 

IS. 

Test  selection  system 

12. 

Less  practical  instruction 

16. 

Computer  and  computer  tester 

13. 

Other  (Mark  No.  13  •  write  on 
comment  sheet) 

Example  of  questionnaire  to  supervisor: 


A.  Never  observed  or  does  not  apply 

B.  Very  weak  In  this  area  (recommend  school  study) 

C.  Weak  In  this  area  (possible  school  problem) 

D.  Satisfactory  performance  (only  a  normal  amount  of  additional  OJT 
required  to  produce  a  field  experienced  repairman) 

E.  Strong  In  this  area  (very  little  additional  OJT  will  be  required) 

F.  Equivalent  to  a  field  experienced  man 

SELECTING  AND  USING  THE  TECHNICAL  MANUALS  APPROPRIATE  TO  THE  JOB  TO  BE  DONE. 


COMPLETING  MAINTENANCE  FORMS  AND  EQUIPMENT  RECORDS  ASSOCIATED  WITH  EACH  JOB. 


SELECTING  AND  USING  SUPPLY  MANUALS  TO  OBTAIN  INFORMATION  NECESSARY  FOR 
REQUISITIONING,  AND  OTHER  SUPPLY  ACTIVITY  ASSOCIATED  WITH  HIS  WORK. 


COMPLETING  SUPPLY  FORMS  AND  RECORDS  ASSOCIATED  WITH  EACH  MAINTENANCE  JOB. 


APPLYING  PRECISION  SOLDERING  TECHNIQUES. 


PROFICIENCY  AND  SUCCESS  IN  ON-SITE  TROUBLESHOOTING! 


GENERAL  OPERATION  OF  FHTE  (TURN-ON,  AOJUSV,  PREPARE  FOR  USE,  SELECT  PATCH- 
CARDS,  BASES,  CORRECT  REFERENCES,  ETC. 


u 


Makes  field  visits.  Sends  questionnaires  to  graduates,  supervisors,  and 
major  commanders. 

Example  of  questionnaire  to  graduate: 


4.  What  job  duties  do  you  have  that  the  course  did  not  tram  you  to  perform? 

5.  Old  the  course  train  you  to  perform  duties  that  are  not  required  on  the 
job?  If  so,  what  are  those  duties? 


Example  of  supervisor  questionnaires: 


1. 

2.  List  the  major  duties  (not  more  than  J.)  of  the  job  presently  being  per¬ 
formed  by  this  man. 

3.  Does  this  man  display  the  technical  knowledge  required  to  perform  satis¬ 
factorily  at  the  MOS  skill  level  for  which  he  was  school  trained?  If  not. 

In  what  technical  areas  1$  he  deficient? 

♦  *  *  *  * 

2. 

Demonstrated  ability 

Unsatisfactory 

to  apply  knowledge 
to  perform  NOS  tasks 

Satisfactory 

Outstanding 

Has  not  performed  this  task  in  current 
assignment 

JOB  WQW  LEDGE  AMO  SKILLS 

NP 

K1 

s 

u 

UNOERSTABS  DUTIES  AMO  RESPONSIBILITIES  OF  ARMY  COOK  MOS  941 . 1 

T1 

■BBBMiiBn^mrpewvnKevmwYfwW'avpvwveOTpBBBBBBHI 

Jr 

H 

m 

1.  Follows  prescribed  procedures  to  prepare  fresh,  frozen, 

— 

B 

dehydrated,  or  canned  foods  for  cooking.  Reference: 

Master  Menu  and  Army  Recipe  Manual. 

1 

2.  Prepares  salad  dressings. 

B 

n 

IMSfffflEiBB 

* 

B 

B 

B 

B 

|  4.  Can  Identify  cuts  of  meat. 

a 

a 

a 

B 

Example  of  questionnaire  to  major  commanJer: 


r——  .  " 1  . - — 

! 

!  a.  QMS  Tng  Objective 

i 

Performance  of 

This  Task  Is 
Performed 
Better  by: 

Remarks 

i 

| 

QMS 

MOS 

Grad 

OJT 

Pers 

t 

|| 

'O 

</>  m 

- 1 
3£ 

p 

i 

i 

j 

r 

! 

1 

i 

j 

PREPARES  FOOD  FOR  COOKING 

OR  SERVING 

1.  Follows  prescribed  procedures 
to  prepare  fresh,  frozen,  de¬ 
hydrated,  or  canned  foods  for 
cooking.  Reference:  Master 
Menu  and  Army  Recipe  Manual. 

- f - 

\ 

■*  ■ 

r  . . 

_ _ 

..... 

! 

( 

2.  Prepares  salad  dressings. 

"  "i 

3.  Prepares  sandwiches. 

'~f  ' 

|  4.  Can  Identify  cuts  of  meat. 

4-L, 

_u , 

_ 

US.ATSCH  (Transportation  School) 

Sends  questionnaires  to  graduates  and  unit  commanders 
Example  of  questionnaire  to  graduate: 

4.  How  wall  did  the  instruction  you  received  at  the  Transportation  School 
prepare  you  for  your  present  duties?  Check  appropriate  block. 

heeded  little  or  no  additional Q  on  the  job  training, schooling. 

Required  sane  additional fH  on  the  job  training, Q  schooling. 

Required  extensive  additional \  ]  on  the  job  training,  FJ  schooling. 

Aw  not  now  performing  duties  for  which  school  trained. f~“l 


I 


Example  of  questionnaire  to  graduate:  (Continued) 


I 

| 

j 

f 

i 


i 


Section  12  need  not  be  completed  If  you  are  not  working  In  the  school 
trained  area. 


SECTION  II 


5. 


DESCRIPTION  OF  CURRENT  What  l  of  your 
DUTIES  tine  oo  you  spend 

In  each  duty? 


MUST  TOTAL  100* 


4- 

I 

4- 


■f 


i 

i 


(Please  check  appropriate  block) 
When  I  performed  this  duty*  I 
found  it: 

FAIRLY  VERY 
EASY  DIFFICULT  DIFFICULT 


i 

* 


This  School  welcomes  any  co— ants  you  nay  desire  to  make  regarding  the 
course  you  attended.  Please  include  consents  on  separate  sheet(s)  and 
return  with  this  questionnaire. 


Example  of  questionnaire  to  commander: 


4.  Compared  to  all  others  who  have 
performed  for  you  on  a  stellar  job: 

EXPLANATION:  1  -  Lowest.  5  *  Highest. 
Other  ratings  represent  variations 
between  the  two  extremes.  Please  circle 
the  appropriate  number. 

a.  How  well  does  he  knew  all 
aspects  of  Ms  specific  job? 

1 

2 

3 

4 

$ 

b.  Haw  well  would  he  function  In 
spacialited  areas  of  his  HOS 
which  he  Is  not  now  performing? 

1 

2 

3 

4 

S 

c.  How  well  did  he  ptrfo  ..  Ms  job 
without  additional  training? 

1 

2 

3 

4 

$ 

d.  Hew  well  dues  he  perform  the 
routine  frictions  of  Ms  job 
without  supervision? 

1 

2 

3 

4 

5 

e.  How  well  does  Ms  training 
assist  hte  in  overcoming  new 
and  different  situations? 

1 

2 

3 

4 

$ 

I 
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Office,  Chief  of  Research  and  Development 

Consolidated  Training  and  Education 

Department  of  the  Army 

Program  (CTEP) 

Washington,  C.C.  20310 

g 

Curriculum  development  procedures  in  use 

as  of  1966  for  first-enlistment 

technical  training  in  the  Army,  Kavy,  and  Air  force  are  analysed.  A  model 
process  for  training  curriculum  development  was  defined  from  training  research 
finding*  and  practices:  (a)  Analyse  the  system,  lb)  develop  task  inventories, 

(c)  develop  a  Job  model.  (d)  analyse  its  tasks,  (#)  derive  training  objectives, 

(t)  develop  the  training  program,  and  (g)  non' tor  the  trained  product  and  modify 
the  curriculum.  A  comparison  between  this  model  and  the  training  development 
procedure*  in  use  in  the  services  indicated  a  need  for  (at  better  procedures 
far  determining  the  adequacy  of  training  content  and  the  means  for  improvement; 

(b)  detailed  guidance  for  developing  or  conducting  the  first  four  steps  of  the 
model  process,  criteria  for  allocating  training  content  tc  formal  instruction 
or  on- the- fob  learning,  performance  specifications  foe  graduates,  and  feedback 
from  training  program*;  and  Cc)  more  opportunities  for  career  fields  in  training. < 
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